ptg16105617

From the Library of YIGUANG HU

ptg16105617

The Go

Programming

Language

From the Library of YIGUANG HU

ptg16105617

This page intentionally left blank

From the Library of YIGUANG HU

ptg16105617

The Go

Programming

Language

Alan A. A. Donovan

Google Inc.

Brian W. Kernighan

Princeton University

New York • Boston • Indianapolis • San Francisco

Toronto • Montreal • London • Munich • Paris • Madrid

Capetown • Sydney • Tokyo • Singapore • Mexico City

From the Library of YIGUANG HU

ptg16105617

Many of the designations used by manufacturers and sellers to distinguish their products are claimed

as trademarks. Where those designations appear in this book, and the publisher was aware of a trade-

mark claim, the designations have been printed with initial capital letters or in all capitals.

 e authors and publisher have taken care in the preparation of this book, but make no expressed

or implied warranty of any kind and assume no responsibility for errors or omissions. No liability is

assumed for incidental or consequential damages in connection with or arising out of the use of the

information or programs contained herein.

For information about buying this title in bulk quantities, or for special sales opportunities (which

may include electronic versions; custom cover designs; and content particular to your business, train-

ing goals, marketing focus, or branding interests), please contact our corporate sales department at

corpsales@pearsoned.com or (800) 382-3419.

For government sales inquiries, please contact governmentsales@pearsoned.com.

For questions about sales outside the United States, please contact international@pearsoned.com.

Visit us on the Web: informit.com/aw

Library of Congress Control Number: 2015950709

and permission must be obtained from the publisher prior to any prohibited reproduction, storage in a

retrieval system, or transmission in any form or by any means, electronic, mechanical, photocopying,

recording, or likewise. To obtain permission to use material from this work, please submit a written

request to Pearson Education, Inc., Permissions Department, 200 Old Tappan Road, Old Tappan, New

Jersey 07675, or you may fax your request to (201) 236-3290.

Front cover: Millau Viaduct, Tarn valley, southern France. A paragon of simplicity in modern engi-

neering design, the viaduct replaced a convoluted path from capital to coast with a direct route over

the clouds. © Jean-Pierre Lescourret/Corbis.

tions 3.0 license.

Typeset by the authors in Minion Pro, Lato, and Consolas, using Go, gro , ghostscript, and a host of

other open-source Unix tools. Figures were created in Google Drawings.

ISBN-13: 978-0-13-419044-0

ISBN-10: 0-13-419044-0

Text printed in the United States on recycled paper at RR Donnelley in Crawfordsville, Indiana.

First printing, October 2015

From the Library of YIGUANG HU

ptg16105617

Fo r Leil a an d Me g

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

This page intentionally left blank

From the Library of YIGUANG HU

ptg16105617

Cont ents

Pr eface xi

TheOrigins ofGoxii

TheGoPro jec t xiii

Organizat ionofthe Bookxv

Wh ere toFindMoreInfor mat ionxvi

Ac knowledgments xvii

1. Tutorial1

1.1. Hel lo, Wor ld1

1.2. Command-L ineArguments 4

1.3. FindingDup lic ateLines 8

1.4. Animated GIFs 13

1.5. FetchingaURL 15

1.6. FetchingURLs Con cur rently17

1.7. A We b Server 19

1.8. Loose End s 23

2. Pro gramStr ucture27

2.1. Names 27

2.2. Declarat ions 28

2.3. Var iables 30

2.4. Assig nments 36

2.5. Typ e Decl arat ions 39

2.6. Packages andFiles 41

2.7. Scope 45

vii

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

viii CONTENTS

3. Basic Data Typ es51

3.1. Int egers 51

3.2. Float ing-Point Numbers 56

3.3. Complex Numbers 61

3.4. Boole ans63

3.5. Str ings64

3.6. Con stants75

4. Com positeTyp es81

4.1. Arrays 81

4.2. Slices 84

4.3. Maps 93

4.4. Str ucts99

4.5. JSON107

4.6. Text andHTMLTempl ates113

5. Func tions 119

5.1. FunctionDeclarat ions 119

5.2. Rec ursion121

5.3. MultipleRetur n Va lues 124

5.4. Erro rs127

5.5. FunctionValues 132

5.6. Anony mou s Func tions 135

5.7. Var iadic Functions 142

5.8. Defer red FunctionCal ls143

5.9. Panic 148

5.10. Recov er151

6. Metho ds 155

6.1. Met hod Declarat ions 155

6.2. Met hodswit h aPoint erReceiver158

6.3. ComposingTyp es by Str uct Emb edding161

6.4. Met hod Values andExpressions 164

6.5. Example: Bit Vec tor Typ e 165

6.6. Encapsulat ion168

7. Interfaces171

7.1. Int erfaces as Contrac ts 171

7.2. Int erface Typ es 174

7.3. Int erface Satisfac tion175

7.4. ParsingFlags wit h flag.Value 179

7.5. Int erface Values 181

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

CONTENTSix

7.6. Sor tingwit h sort.Interface 186

7.7. The http.Handler Interface 191

7.8. The error Interface 196

7.9. Example: ExpressionEvaluator197

7.10. Typ e As ser tions 205

7.11. Discriminat ingError s with Typ e As ser tions 206

7.12. Quer yingBeh avior s with Int erface Typ e As ser tions 208

7.13. Typ e Sw itch es210

7.14. Example: Token-B ased XML Decoding213

7.15. A Fe w Wo rds ofAdv ice 216

8. Gor o utines and Channels 217

8.1. Goroutines217

8.2. Example: Con cur rentClo ckSer ver 219

8.3. Example: Con cur rentEch o Server 222

8.4. Channel s 225

8.5. Looping inParal lel234

8.6. Example: Con cur rentWeb Craw ler 239

8.7. Multiplexingwit h select 244

8.8. Example: Con cur rentDirec tor y Traversal247

8.9. Cancellat ion251

8.10. Example: ChatSer ver 253

9. Concurrency withShared Vari ables257

9.1. Race Con dit ion s 257

9.2. Mut ual Exc lusion: sync.Mutex 262

9.3. Read/Write Mut exes: sync.RWMutex 266

9.4. Memor y Sy nchro nizat ion267

9.5. Lazy Initializat ion: sync.Once 268

9.6. TheRace Detec tor 271

9.7. Example: Con cur rentNon-Blo cking Cache 272

9.8. Goroutinesand Threads 280

10. Pack ages and the GoTool283

10.1. Int roduc tion283

10.2. Imp ort Pat hs284

10.3. ThePackageDeclarat ion285

10.4. Imp ort Declarat ions 285

10.5. Blank Imp orts286

10.6. Packages andNaming289

10.7. TheGoTool290

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

xCONTENTS

11. Testing 301

11.1. The go test To ol302

11.2. Test Func tions 302

11.3. Cov erage318

11.4. Benchmark Func tions 321

11.5. Proﬁling323

11.6. Example Func tions 326

12. Reﬂecti on329

12.1. Why Reﬂec tion?329

12.2. reflect.Type and reflect.Value 330

12.3. Display,aRec ursiveValue Print er333

12.4. Example: Enco dingS-E xpressions 338

12.5. Setting Var iables wit h reflect.Value 341

12.6. Example: DecodingS-E xpressions 344

12.7. AccessingStr uct Field Tags 348

12.8. Displaying the Met hodsofaTyp e 351

12.9. A Wo rdofCaution 352

13. Low-L evel Pro gramming353

13.1. unsafe.Sizeof, Alignof,and Offsetof 354

13.2. unsafe.Pointer 356

13.3. Example: DeepEquivalence 358

13.4. Cal lingCCodewit h cgo 361

13.5. Another WordofCaution 366

Index367

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

Pref ace

‘‘Go is anopensourc e prog rammi n g lang uagethatmak esiteasytobui ldsimpl e,rel iab le,

an d efﬁcientsof tware.’’ (Fro m theGoweb sit e at golang.org)

Go was conceive d in Septemb er2007 byRob ert Gries emer,Rob Pike, and Ken Thomps on, all

at Google, and was announced inNov emb er2009. Thegoals ofthe langu ageand its accom-

pany ing tools weretobeexpressive, efﬁcient in bot h comp ilation and exe cut ion,and effec tive

in writing reliableand robustprograms.

Go bears a sur face simi lar ity toC and,likeC,isatoolfor prof essionalprogrammers, achie v-

ingmaximum effe ctwit h minimum means.But it ismuchmorethananupdated versionof

C. Itbor rowsand adaptsgood ide as from manyother langu ages, whi le avoidingfeaturesthat

have led tocomplexity and unreliablecode. Its facilities for con cur rency are new and efﬁcient,

andits approach to dat a abstrac tionand obj e ct-oriente d prog rammingisunu sually ﬂexible. It

hasaut omat ic memory managementor garb age col lec tion.

Go isesp eci ally wel l suit edfor bui ldinginf rastr ucturelikenet wor ked ser vers, andtools and

systems for prog rammers, but it istruly a general-pur pos e language and ﬁnd s us e in domains

as divers e as graphics, mobileapp lic ations,and machinelearning. Ithas becom e popu lar as a

repl acementfor unt ypedscr ipt ing langu ages because itbal ances expressivenesswit h safety :

Go programstypic ally run fasterthanprogramswritt enindynamic langu ages andsuf fer far

fe wer crashesdue tounexp ected typ e er ror s.

Go isanopen-s ource pro jec t,sosourcecodefor itscompi ler,librar ies, andtools is fre ely avai l-

able toany one.Contr ibution s to the pro jec t come fro m an active worldw ide community.Go

runs onUnix-li kesystems—Linux, Fre eBSD, OpenBSD, Mac OS X—andonPlan9and

Micros oft Windows. Programswritt eninone ofthese env iro nmentsgeneral lywor k without

mo diﬁcation on the others.

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

xii PREFACE

Thisbookismeant tohelpyou start usingGoeffec tive lyrig htaway andtouse itwel l,tak ing

full advantage ofGo’slangu agefeaturesand stand ard librar ies to write clear, idiomatic, and

efﬁcientprograms.

TheOrigins ofGo

Like biolog ical species, successf ullangu ages beget offspring thatincor poratethe advantagesof

their ancestors;int erbre e dingsom etimesleads tosur prisingstrengt hs; and, ver y occasionally,

aradic al ne w fe ature ariseswit houtpre cedent. Wecan lear n alot about why a langu ageisthe

way itisand whatenv iro nment ithas beenadapt edfor bylooking atthese inﬂuences.

The ﬁgurebelow shows the most imp ortantinﬂuences ofearlier programminglangu ages on

thedesig n of Go.

Go issom etimesdes crib edasa‘‘C-li kelangu age,’’ or as ‘‘Cfor the 21stcentury.’’ Fr omC,Go

in her ite d itsexpressionsyntax, cont rol-ﬂow statements, basic dat a types, cal l-by-value param-

eter passing, point ers,and above all,C’semp hasisonprogramsthatcompi letoefﬁcient

machinecodeand cooperatenatural lywit h theabstrac tions ofcur rentoperat ingsystems.

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

THE ORIGINS OF GO xiii

Butthere are other ancestors inGo’sfami lytre e.One maj orstreamofinﬂuence comesfro m

languagesbyNik lausWir th, beg inningwit h Pascal.Modula-2 inspired the packagecon cept.

Ob ero n eliminated the distinc tionbet weenmoduleint erface ﬁles andmoduleimp lementation

ﬁles. Obero n-2 inﬂuence d thesyntaxfor packages, imports, anddeclarat ions,and Obj e ct

Ob ero n prov ide d thesyntaxfor met hod declarat ions.

Anot her lineage among Go’sancestors,and one thatmakes Godistinc tiveamong recent

prog ramminglangu ages, isa sequence oflit tle-k now n research langu ages deve lop edatBel l

Labs, allins pired bythe con ceptof commu nicating sequent ial pro cesses (CSP) fro m To ny

Ho are’sseminal 1978 pap er on the found ation s of con cur rency.InCSP,aprogram isaparal lel

comp osition of processesthathavenoshare d st ate;the pro cessescommunic ateand synchro-

nize usingchannel s.But Hoare’sCSP was a for mal langu agefor des cribingthe fundament al

concepts ofcon cur rency,not a programminglangu agefor writing exe cut ableprograms.

RobPikeand othersbegan toexp erimentwit h CSPimp lementation s as actu allangu ages. The

ﬁrs t was cal le d Sque ak (‘‘Alangu agefor communic atingwit h mice’’), whichprovide d alan-

gu agefor handlingmou seand key board events, wit h st aticallycre ate d ch annel s.Thiswas

fo llowe d by Newsque ak, whichoffered C-li kestatement and expressionsyntaxand Pas cal-li ke

type not ation.Itwas a purelyfunctionallangu agewit h garb agecol lec tion,again aimed at

managing key board , mous e,and windowevents. Channel s became ﬁrs t-class values, dynami-

callycre ate d andstorable in variables.

ThePlan9operat ingsystemcar r iedthese ide as forwardinalangu agecal le d Alef.Alef tried

to makeNewsque ak aviablesystemprogramminglangu age, but itsomissionofgarb agecol-

le ction made conc urrency too painf ul.

Ot her cons tructions inGoshowthe inﬂuence ofnon-ancestral genes hereand there;for

example iota is loosely fro m APL, andlexic al scop e with neste d func tions isfro m Scheme

(andmostlangu ages since). Heretoo weﬁnd nove l mu tat ions.Go’sinnovat ive slices provide

dy namic arrays wit h efﬁcientrandomaccessbut als o permit sop histicate d sh aring

ar rangementsreminiscentoflin ked lists. And the defer st atement isnew wit h Go.

TheGoProject

Al l prog ramminglangu ages reﬂec t theprogrammingphi losop hyoftheir creators,whichoften

includes a signiﬁcant component ofreactiontothe perceive d shortcomings ofearlier lan-

gu ages. TheGopro jec t was bor neoffrust rat ionwit h several sof twaresystems atGooglethat

were suf fer ingfro m an explosionofcomplexity.(Thispro blem isbynomeans unique to

Go ogle.)

As Rob Pikeput it, ‘‘comp lexity ismultiplic ative’’: ﬁxingapro blem bymak ingone par t of the

systemmorecomplex slowlybut surelyaddscomplexity toother par ts. Wi thcon stant pres-

sure toadd featuresand opt ion s andconﬁgurat ions,and toshipcodequickly, it’seasy to

neglec t simplicity,eventhoug h in the lon g runsimplicity isthe key togood sof tware.

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

xiv PREFACE

Simp licity requires morewor k at the beg inningofapro jec t to reduce an ideatoits essenceand

more dis cip lineoverthe lifet imeofapro jec t to distinguish good changesfro m badorper ni-

cious ones. Wit h sufﬁcienteffor t,agood change can beaccommodated wit houtcompromis-

ingwhatFre d Brooks cal le d the ‘‘conceptu alint egr ity’’ of the desig n butabad change cannot,

andaper nicious change trades simplicity for itsshallow cou sin, conv enience.Onlythrough

simplicity ofdesig n canasystemremain stable, sec ure , andcoh erent asitgrows.

TheGopro jec t includes the langu ageits elf,its tools and stand ard librar ies, andlastbut not

le ast, a cultural agenda ofradic al simplicity.Asarecenthig h-leve l language , Go has the bene-

ﬁtofhindsig ht, andthe basics aredon e we ll: ithas garbagecol lec tion,apackagesystem, ﬁrs t-

cl ass functions,lexic al scop e,asystemcal l interface,and immut ablestr ingsinwhichtext is

general lyencoded inUTF-8. But it has comp arat ive lyfew featuresand isunlikelytoadd

more . Fo r inst ance, ithas noimp licitnumer ic conv ersions,nocon str uctor s or destr uctor s,no

op erator overloading, nodefau ltparameter values, noinher itance, nogener ics, no exception s,

no macros, nofunctionannot ation s,and no thread-lo cal storage. The langu ageismatureand

st able, and guarante esbackwards compatibi lit y:older Goprogramscan becompi led and run

with newer versions ofcompi lersand stand ard librar ies.

Go has enough ofatyp e systemtoavoid mostofthe carelessmistakesthatplague program-

mers indynamic langu ages, but it has a simpler typ e systemthancomparable typ edlangu ages.

Thisappro ach can som etimesleadtoisolate d pockets of ‘‘untyped’’ prog rammingwit hin a

broader framewor k of typ es, andGoprogrammersdonot gotothe lengt hsthatC++ or

Haskel l prog rammersdotoexpress safet y prop erties as typ e-bas edpro ofs. But inprac tice Go

givesprogrammersmuchofthe safet y andrun-t imeper for mance beneﬁts ofarel ative ly

st ron g type systemwit houtthe burden ofacomplex one.

Go encourages an awarenessofcontemp orar y comp utersystemdesig n, partic ularlythe

importance oflocality.Its bui lt-in dat a typesand most librar y data str uctures arecraf ted to

work natural lywit houtexplicitinitializat ionorimp licitcon str uctor s,sorel ative lyfew mem-

or y al location s andmemor y wr itesare hidden in the code. Go’saggregatetyp es (str uctsand

ar rays) hold their elements direc tly,requir inglessstorageand fewer allo cat ions and point er

indirec tion s than langu ages thatuse indirec t ﬁelds.And since the moder n comp uterisapar-

al lelmachine, Gohas conc urrency featuresbas edonCSP,asmention edearlier.The var iable-

size stacksofGo’slig htweig htthreads or goro utines areinitial lysmall enoug h that creating one

goro utine ische apand cre ating a millionisprac tic al.

Go’sstand ard librar y,often descr ibedascomingwit h ‘‘batt eries include d,’’ prov ides cle an

buildingblo cks andAPIsfor I/O,text pro cessing, graphics, cryptography,net wor king, and

dist ribut edapp lic ations,wit h supp ort for manystand ard ﬁle for mats andpro tocol s.The

librar ies andtools make extensive use ofconvent ion toreduce the need for conﬁgurat ionand

explanation,thu s simplif yingprogram logic andmak ingdiverse Go programsmoresimi lar to

each other andthu s easier tolearn.Pro jec tsbui ltusingthe go to oluse onlyﬁle andidentiﬁer

namesand anocc asionalspeci al commenttodeter mineall the librar ies, exec utables, tests,

benchmarks, examples, platfor m-speciﬁcvar iants, anddocumentation for a proj e ct; the Go

source its elf cont ainsthe bui ld sp eciﬁcation.

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

THE GO PROJECTxv

Or ganization ofthe Book

We assume thatyou haveprogrammed inone ormoreother langu ages, whether comp ile d li ke

C, C++, andJava, orint erprete d li kePyt hon,Ruby, and JavaS crip t,sowewon’tspell out every-

thing as if for a tot albeg inner.Sur face sy ntaxwill befami liar, as will var iables andcon stants,

expressions,control ﬂow,and functions.

Chapter1 isatut orialonthe basic cons tructsofGo, int roduce d thro ugh a dozen programsfor

ever ydaytasks like readingand writing ﬁles, for matting text, creating images, andcommuni-

cating wit h Internet clients andser vers.

Chapter2 des crib esthe str uctural elements ofaGoprogram—de clarat ions,var iables, new

types, packages and ﬁles, andscope.Chapt er3 dis cussesnumbers,boole ans, str ings, andcon-

st ants, andexplainshow topro cessUnico de.Chapt er4 des crib escomposite typ es, that is,

typesbui ltupfro m simpler onesusingarrays, maps, str ucts, and sli ces,Go’sappro ach to

dy namic lists. Chapt er5 cov ers functions and dis cusseserror handling, panic and recover,

andthe defer st atement.

Chapters 1 through 5 are thu s thebasics, things thatare par t of any mainst reamimp erat ive

language . Go’ssyntaxand sty lesom etimesdif fer fro m ot her langu ages, but mostprogram-

mers will pickthemupquickly. The remainingchapt ers focus ontopics where Go’sappro ach

is lessconvent ion al: met hods, interfaces, conc urrency,packages, testing , andreﬂec tion.

Go has an unusualappro ach to obj e ct-oriente d prog ramming. There are noclass hierarchies,

or indeed any class es; comp lex obj e ctbeh avior s arecre ate d from simpler onesbycomposition,

notinher itance. Met hodsmay beass oci ated wit h anyuser-deﬁne d type,not juststr uctures,

andthe rel ation shipbet weencon crete typ es andabstrac t types(interfaces)isimp licit, soa

concrete typ e maysat isf y an interface thatthe typ e’sdesig ner was unawareof. Met hodsare

covere d in Chapt er6and int erfaces in Chapt er7.

Chapter8 presentsGo’sappro ach to con cur rency,whichisbas edonthe ide a of communic at-

ingsequential pro cesses(CSP), embodie d by goroutinesand channel s.Chapt er9 explainsthe

more tradition alasp ectsofcon cur rency bas edonshare d var iables.

Chapter10des crib espackages, the mech anism for organizinglibrar ies. Thischapt erals o

shows how tomakeeffec tive use ofthe go to ol, whichprovides for compi lat ion, testing ,

benchmarking , prog ram formatting , do cumentation,and manyother tasks, allwit hin a single

command.

Chapter11deals wit h test ing , whereGotakes a not ably lig htweig htappro ach,avoiding

abstrac tion-l aden framewor ksinfavor of simplelibrar ies andtools.The testing librar ies

prov ide a found ation atopwhichmorecomplex abstrac tions can bebui lt if necessary.

Chapter12dis cussesreﬂec tion,the abi lit y of a prog ram to examineits own represent ation

during exe cut ion.Reﬂec tion isapow erfultool, thoug h on e to beusedcaref ully; thischapt er

explains ﬁndingthe rig htbal ance byshowing how itisusedtoimp lementsom e importantGo

librar ies. Chapter13explainsthe gor y det ails oflow-le vel programmingthatusesthe unsafe

packagetosteparo und Go’styp e system, andwhenthatisappro priate.

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

xvi PREFACE

Each chapterhas a numberofexercis esthatyou can use totestyourunderst andingofGo, and

to explore extensions and alt ernat ivestothe examples fro m thebook.

Al l butthe most trivialcodeexamples in the bookare avai lable for dow nlo ad from the public

Gitrep ository at gopl.io.Eachexampleisidentiﬁed byits packageimp ort pat h andmay be

conv enientlyfetch ed, bui lt, andins tal le d usingthe go get command. You’ll need tocho ose a

direc tor y to beyourGowor ksp ace andset the GOPATH enviro nment var iable topoint toit.

The go to olwill cre ate the direc tor y if necessary.For example:

$export GOPATH=$HOME/gobook #choose workspace directory

$goget gopl.io/ch1/helloworld #fetch, build, install

$$GOPATH/bin/helloworld #run

Hello, BF

To r un the examples, you will need atleast version1.5 ofGo.

$goversion

go version go1.5 linux/amd64

Fo llowthe ins tructions at https://golang.org/doc/install if the go to olonyourcom-

puterisolder ormissing.

WheretoFind MoreInformation

Thebestsourcefor moreinfor mat ionabout Goisthe ofﬁci al we b site, https://golang.org,

whichprovides accesstothe documentation,includingthe Go Programmi n g Lang uageSpeciﬁ-

cati on,stand ard packages, andthe like. There are als o tutorialsonhow towrite Goand how

to write itwel l,and a wide variet y of onlinetext andvide o resources thatwill bevaluablecom-

plements tothisbook. TheGoBlog at blog.golang.org pu blishessom e of the bestwriting

on Go, wit h ar ticles onthe state ofthe langu age, plans for the fut ure , reportsonconferences,

andin-depth explanation s of a wide variet y of Go-rel ate d topics.

Oneofthe most usefulasp ectsofonlineaccesstoGo(andareg rettablelimitation of a pap er

book) isthe abi lit y to run Goprogramsfro m theweb pages thatdes crib e them. Thisfunc-

tion ality isprovide d by the GoPlayg round at play.golang.org,and may beemb edde d

within other pages, suchasthe home pageat golang.org or the documentation pages ser ved

by the godoc to ol.

ThePlayg round makes itconvenienttoper for m simpleexp eriments toche ckone’sunder-

st andingofsyntax, semantics, orlibrar y packages wit h short programs, andinmanyways

takesthe place ofare ad-e val-print loop (REPL) in other langu ages. Its persistentURLs are

greatfor sharing snipp ets ofGocodewit h ot hers, for rep ortingbugs ormak ingsug gestion s.

Builtatopthe Playg round,the GoTourat tour.golang.org is a sequence ofshort int erac tive

lessons onthe basic ideas andcon str uctions ofGo, anorderly wal k thro ugh the langu age.

Theprimary shortcomingofthe Playg round and the Touristhatthe y al lowonlystand ard

librar ies to beimp orted,and manylibrar y fe atures—networ king, for example—arerestr icted

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

WHERE TOFIND MORE INFORMATION xvii

forprac tic al or sec urity reasons.The y also requireaccesstothe Int ernet tocompi leand run

each program. Sofor moreelaborateexp eriments, you will havetorun Goprogramsonyour

ow n comp uter. For tunatelythe dow nlo ad processisstraig htfor ward, soitshouldnot take

more thanafew minut estofetch the Godistr ibution fro m golang.org andstart writing and

runningGoprogramsofyourown.

SinceGoisanopen-s ource pro jec t,you can readthe codefor any typ e or function in the stan-

dard librar y on lineat https://golang.org/pkg;the samecodeispar t of the dow nlo ade d

dist ribut ion.Use thisto ﬁgureout how som ethingwor ks, ortoanswerquestion s ab out

det ails,ormerelytosee how exp erts write reallygood Go.

Acknowledgments

RobPikeand RussCox,coremembers ofthe Goteam, readthe manus cript several times wit h

greatcare; their comments oneverythingfro m word choice tooveral l st ruc tureand organiza-

tion havebeeninvaluable. Whi le prep aringthe Jap anes e transl ation,Yoshi kiShibata wentfar

beyond the cal l of duty; his met iculou s eyespott ednumerou s inconsistencies in the Eng lish

text anderror s in the code. Wegre atlyappre ciate thoro ugh revie ws andcriticalcommentson

theent ire manus cript fro m Br ian Goetz, Corey Kos ak, Ar noldRobbins, JoshBle e cherSny der,

andPeter Weinberger.

We a re indebte d to SameerAjmani,Itt aiBal aban, Dav id Crawshaw, BillyDon ohue, Jon athan

Feinberg , Andrew Gerrand, Rob ert Gries emer,JohnLinder man, Minux Ma, Bryan Mills, Bal a

Nataraj an, Cosmos Nicolaou , Paul Stanifor th, Nigel Tao,and HowardTrickey for many

helpf ulsug gestion s.Weals o than k DavidBrai lsfordand RaphLevien for typ esetting adv ice.

OureditorGregDoenchatAddison-Wesle y gotthe bal l ro llingoriginallyand has beencon-

tinuou sly helpf uleversince.The AWpro duc tionteam—Jo hnFuller,Day naIsley,Julie Nahi l,

Chut i Pras ertsith,and Barb ara Wo o d—hasbeenoutst anding; author s couldnot hop e forbet-

tersup por t.

AlanDon ovan wishestothank:SameerAjmani,Chr isDemet riou, WaltDrummon d,and Reid

TatgeatGooglefor allow ing him timetowrite;Steph enDon ovan, for his adv ice andtimely

encouragement; andabove all,his wifeLei laKazemi,for her unhesitating ent husi asm and

unwaver ingsup por t forthispro jec t,despit e thelon g hoursofdistrac tionand abs ente eism

from fami lylifethatitent ailed.

Br ian Ker nighanisdeeply gratef ultofriends and col leagues for their pat ience andforbearance

as hemov edslowlyalong the pat h to underst anding, and esp eci ally tohis wifeMeg , whohas

been unfailinglysup por tiveofbook-w rit ing and somuchels e.

Ne w Yo r k

Oc tob er2015

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

This page intentionally left blank

From the Library of YIGUANG HU

ptg16105617

Tutorial

Thischapt erisatourofthe basic comp onentsofGo. Wehop e to provide enough infor mat ion

andexamples toget you off the gro und and doing usefulthings as quicklyaspossible. The

examples here, and indeed inthe whole book, areaimed attasks thatyou mig hthavetodoin

therealwor ld. Inthischapt erwe’ll trytogiveyou a taste ofthe diversity ofprogramsthatone

mig htwrite inGo, ranging fro m simpleﬁle pro cessingand a bit of graphics tocon cur rent

Internet clients andser vers. Wecer tain lywon’texplain everythinginthe ﬁrs t ch apt er, but

studying suchprograms in a new langu agecan be an effec tive way toget started.

Wh enyou’re learninganew langu age, there’sanatural tendency towrite codeasyou wou ld

have writt enitinalangu ageyou already know. Beawareofthisbiasasyou learn Goand try

to avoid it. We’ve tried toillustrateand explain how towrite good Go, souse the codehereas

aguide whenyou’re writing your own.

1.1. Hello, World

We’ll start wit h thenow-t radition al ‘‘hel lo, wor ld’’ example, whichapp earsatthe beg inningof

TheCProgrammi n g Lang uage,publishe d in 1978. Cisone ofthe most direc t inﬂuences on

Go,and ‘‘hel lo, wor ld’’ illustrates a numberofcentral ideas.

gopl.io/ch1/helloworld

package main

import "fmt"

func main() {

fmt.Println("Hello, BF")

}

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

2CHAPTER 1. TUTORIAL

Go isacompi led langu age. The Gotoolchain conv ertsasourceprogram andthe things it

dep ends onint o inst ruc tions inthe nat ive machinelangu ageofacomputer. These tools are

accessedthrough a singlecommand cal le d go that has a numberofsub command s.The sim-

plestofthese sub command s is run,whichcompi les the sourcecodefro m on e or moresource

ﬁles whose names endin .go,lin ks it wit h librar ies, then runsthe resulting exe cut able ﬁle.

(Wewill use $ as the command pro mpt throughoutthe book.)

$gorun helloworld.go

No t surprisingly, thisprints

Hello, BF

Go nat ive lyhandles Unico de,soitcan pro cesstext in allthe wor ld’slangu ages.

If the program ismorethanaone-shotexp eriment, it’slikelythatyou wou ldwanttocompi le

it once andsavethe compi led resultfor later use.Thatisdon e with go build:

$gobuild helloworld.go

Thiscre atesanexe cut ablebinar y ﬁle cal le d helloworld that can berun anytimewit houtfur-

ther pro cessing:

$./helloworld

Hello, BF

We havelab ele d each sig niﬁcant exampleasareminder thatyou can obt ain the codefro m the

book’ssourcecoderep ository at gopl.io:

gopl.io/ch1/helloworld

If you run go get gopl.io/ch1/helloworld,itwill fetch the sourcecodeand place itinthe

correspondingdirec tor y.There’smoreabout thistopic in Sec tion 2.6 andSec tion 10.7.

Let’snow tal k ab out the program its elf.Gocodeisorganize d into packages, whichare simi lar

to librar ies or modules in other langu ages. A packagecon sists ofone ormore .go source ﬁles

in a singledirec tor y that deﬁne whatthe packagedoes. Eachsource ﬁle beg inswit h a package

de clarat ion, here package main,thatstateswhichpackagethe ﬁle belon gsto, fol low edbyalist

of other packages thatitimp orts, andthenthe declarat ions ofthe program thatare store d in

that ﬁle.

TheGostand ard librar y hasover100 packages for commontasks like inp utand out put,

sorting, and text manipu lat ion. For ins tance,the fmt packagecontainsfunctions for printing

formatte d output and scanninginp ut. Println is one ofthe basic out put functions in fmt;it

pr intsone ormorevalues, sep arated byspaces, wit h anewlinecharac ter at the end sothatthe

values appear as a singlelineofout put.

Package main is speci al.Itdeﬁnesastand alone exe cut ableprogram, not a librar y.Wit hin

package main the func tion main is als o sp eci al—it’swhere exe cut ion of the program beg ins.

Wh ate ver main do es is whatthe program does. Of course, main wi l l normal lycal l up on func-

tion s in other packages to do muchofthe wor k,such as the function fmt.Println.

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

SECTION 1.1. HELLO, WORLD 3

We musttel l thecompi ler whatpackages areneeded bythissourceﬁle; that’sthe roleofthe

import de clarat ionthatfol lowsthe package de clarat ion. The ‘‘hel lo, wor ld’’ prog ram us es

on lyone functionfro m on e ot her package, but mostprogramswill imp ort morepackages.

Yo u mu stimp ort exac tly the packages you need.Aprog ram wi l l notcompi leifthere are

missingimp ortsorifthere are unnecessary ones. Thisstr ict requirementpre vents references

to unu sed packages fro m acc umulat ing as programsevo l ve.

The import de clarat ions mustfol low the package de clarat ion. After that, a program consists

of the declarat ions offunctions,var iables, cons tants, andtyp es (introduce d by the key words

func, var, const,and type); for the most par t, theorder ofdeclarat ions doesnot matter. This

prog ram is about asshort aspossiblesince itdeclaresonlyone function, whichintur n calls

on lyone other function. Tosavespace,wewill som etimesnot showthe package and import

de clarat ions whenpresent ing examples, but the y areinthe sourceﬁle andmustbethere to

comp ile the code.

Afunctiondeclarat ioncon sists ofthe key word func,the nameofthe function, a parameter

list (empt y for main), a resultlist(also emp tyhere), andthe bodyofthe function—thestate-

mentsthatdeﬁne whatitdoes—enclos edinbraces. We’ll takeaclos er lo okatfunctions in

Chapter5.

Go doesnot requiresemicolons atthe end s of statementsordeclarat ions,exceptwhere two or

more app ear onthe sameline. Ineffec t,newlines fol low ing cer tain tokensare converted int o

semicolons,sowhere newlines areplace d matt ers to pro per parsingofGocode. For ins tance,

theopeningbrace { of the functionmustbeonthe samelineasthe end ofthe func de clara-

tion,not onalinebyits elf,and inthe expression x+y,anewlineisper mitt edafter but not

before the + op erator.

Go takes a stron g st anceoncodefor matting . The gofmt to olrewritescodeint o thestand ard

format, andthe go to ol’s fmt su bcommandapp lies gofmt to all the ﬁles in the speciﬁed pack-

age, orthe onesinthe cur rentdirec tor y by defau lt. All Gosourceﬁles in the bookhavebeen

runthrough gofmt,and you shouldget into the habit of doing the samefor yourown code.

Decl aring a stand ard for mat by ﬁat eliminates a lotofpoint lessdeb ate about trivia and,more

importantly, enables a variet y of aut omated sourcecodetransfor mat ions thatwou ldbe

infeasible if arbit rar y formatting were allow ed.

Many text editors can beconﬁgured torun gofmt each timeyou saveaﬁle,sothatyoursource

co de is always pro perly for matte d. Arel ate d to ol, goimports,addition allymanages the ins er-

tion and removal ofimp ort declarat ions asneeded.Itisnot par t of the stand ard distr ibution

butyou can obt ain itwit h this command:

$goget golang.org/x/tools/cmd/goimports

Fo r most users,the usu alway todow nlo ad andbui ld packages, run their tests, showtheir doc-

umentation,and soon, iswit h the go to ol, whichwe’ll look at in Sec tion 10.7.

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

4CHAPTER 1. TUTORIAL

1.2. Command-Line Arguments

Most programspro cesssom e inputtopro duce som e output;that’sprett y much the deﬁnition

of computing . Buthow doesaprogram get inputdat a on whichtooperate? Som e prog rams

generatetheir own data,but moreoften, inputcom esfro m an exter nal source: a ﬁle,anet wor k

connec tion,the out put of another program, a useratakey board , command-linearguments,

or the like. The next few examples will dis cusssom e of these alt ernat ives, startingwit h com-

mand-linearguments.

The os packageprovides functions and other values for dealingwit h theoperat ingsystemina

pl atfor m-indep endentfashion. Command-linearguments areavai lable toaprogram in a

var iable named Args that ispar t of the os package; thu s itsnameany where outside the os

packageis os.Args.

Thevar iable os.Args is a sli ce of str ings. Slices areafundament alnot ion inGo, and we’ll tal k

alot moreabout themsoon. For now,thin k of a slice as a dynamic ally size d sequence s of

ar ray elements where indiv idu alelements can beaccessedas s[i] andacontiguous subse-

quence as s[m:n].The numberofelements isgiven by len(s).Asinmostother program-

minglangu ages, allindexinginGouses half-open intervalsthatinclude the ﬁrs t index but

exclude the last, because itsimpliﬁes logic. For example, the slice s[m:n],where 0 ≤ m ≤ n ≤

len(s),contains n-m elements.

The ﬁrs t elementof os.Args, os.Args[0],isthe nameofthe command its elf; the other ele-

mentsare the arguments thatwerepresent edtothe program whenitstarted exe cut ion.A

slice expressionofthe for m s[m:n] yields a slice thatreferstoelements m thro ugh n-1,sothe

elements weneed for our next exampleare those inthe slice os.Args[1:len(os.Args)].If m

or n is omitt ed, itdefau lts to0or len(s) resp ectively, sowecan abbreviate the desired slice as

os.Args[1:].

Here’sanimp lementation of the Unix echo command, whichprintsits command-lineargu-

mentsonasingleline. Itimp ortstwo packages, whichare given as a parenthesize d list rat her

than as individu al import de clarat ions.Eit her for m is legal,but convent ion allythe listfor m is

us ed. The order ofimp ortsdoesn’tmatter; the gofmt to olsor tsthe packagenames into

alph abeticalorder.(Wh enthere are several versions ofanexample, wewill often number

them soyou can besureofwhichone we’re tal kingabout.)

gopl.io/ch1/echo1

// Echo1 prints its command-line arguments.

package main

import (

"fmt"

"os"

)

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

SECTION 1.2. COMMAND-LINE ARGUMENTS5

func main() {

var s, sep string

for i := 1; i < len(os.Args); i++ {

s+=sep + os.Args[i]

sep = " "

}

fmt.Println(s)

}

Comments beg in with //.All text fro m a // to the end ofthe lineiscomment ary for

prog rammersand isignored bythe compi ler.Byconvent ion,wedes crib e each package in a

commentimmediate lypre cedingits packagedeclarat ion; for a main package, thiscomment is

on e or morecompletesentences thatdes crib e theprogram as a whole.

The var de clarat iondeclarestwo var iables s and sep,oftyp e string.Avar iable can beini-

tialize d as par t of itsdeclarat ion. Ifitisnot explicitlyinitialize d,itisimp licitlyinitialize d to

the zero value forits typ e,whichis 0 fornumer ic typesand the emp tystr ing "" forstr ings.

Thus inthisexample, the declarat ionimp licitlyinitializes s and sep to emp tystr ings. We’ll

have moretosay aboutvar iables anddeclarat ions inChapt er2.

Fo r numb ers,Goprovides the usu alarithmeticand log ical operator s.Whenapp lie d to

st rings, how ever, the + op erator concatenate s thevalues, sothe expression

sep + os.Args[i]

repres ents the con catenat ionofthe str ings sep and os.Args[i].The statement weusedin

theprogram,

s+=sep + os.Args[i]

is an assig nment statement that conc atenatesthe old value of s with sep and os.Args[i] and

assig nsitbackto s;itisequivalentto

s=s+sep + os.Args[i]

Theoperator += is an assig nment operator.Eacharithmeticand log ical operator like + or * has

acor respondingassig nmentoperator.

The echo prog ram couldhaveprint edits out put inaloopone pie ce at a time, but thisversion

insteadbui ldsupastr ing byrep eatedlyapp endingnew text tothe end.The str ing s st artslife

empt y,thatis, wit h value "",and eachtripthrough the loopaddssom e text toit; afterthe ﬁrs t

it erat ion, a space isals o inserted sothatwhenthe loopis ﬁnishe d,there isone space bet ween

each argument. Thisisaquadrat ic processthatcou ldbecostlyifthe numberofarguments is

large , butfor echo,that’sunlikely. We’ll showanumberofimp rov edversions of echo in this

ch apt erand the next thatwill dealwit h anyrealinefﬁciency.

Theloopindex variable i is declare d in the ﬁrs t part ofthe for lo op. The := sy mbolispar t of

a sh ort var iab ledeclarati on,astatement thatdeclaresone ormorevar iables andgives them

appropriatetyp es basedonthe initializer values; there’smoreabout this in the next chapt er.

Theincrement statement i++ adds1to i;it’sequivalentto i+=1whichisintur n equivalent

to i=i+1.There’sacor respondingdecrement statement i-- that subtrac ts 1. Thes e are

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

6CHAPTER 1. TUTORIAL

st atements, not expressions asthe y areinmostlangu ages in the C fami ly, so j=i++ is illegal,

andthe y arepostﬁxonly, so --i is not legal either.

The for lo opisthe onlyloopstatement inGo. Ithas a numberoffor ms, one ofwhichis

illustrated here:

for

initialization

;

condition

;

post

{

// zero or more statements

}

Parent hes es arenever usedaro und the three componentsofafor lo op. The braces are

mand atory,how ever, and the openingbrace mustbeonthe sameline as the

post

st atement.

Theopt ion al

initialization

st atement isexe cut edbeforethe loopstarts. Ifitispresent,it

mu stbeasimp lestatement,thatis, a short var iable decl arat ion, an incrementorassig nment

st atement,orafunctioncal l.The

condition

is a boole an expressionthatisevaluate d at the

beginningofeachiterat ionofthe loop; if itevaluatesto true,the statementscontrol led bythe

lo opare exe cut ed. The

post

st atement isexe cut edafter the bodyofthe loop, thenthe con di-

tion isevaluate d again. Theloopend s when the con dit ion becom esfalse.

Anyofthese par ts maybeomitt ed. Ifthere isno

initialization

andno

post

,the semi-

colons may also beomitt ed:

// a traditional "while" loop

for

condition

{

// ...

}

If the con dit ion isomitt edent ire ly in any ofthese for ms, for examplein

// a traditional infinite loop

for {

// ...

}

theloopisinﬁnite, thoug h lo ops ofthisfor m maybeter minated insom e ot her way,likea

break or return st atement.

Anot her for m of the for lo opiterates overara nge of values fro m adat a type likeast ringora

slice.Toillustrate, here’sasecon d versionof echo:

gopl.io/ch1/echo2

// Echo2 prints its command-line arguments.

package main

import (

"fmt"

"os"

)

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

SECTION 1.2. COMMAND-LINE ARGUMENTS7

func main() {

s, sep := "", ""

for _, arg := range os.Args[1:] {

s+=sep + arg

sep = " "

}

fmt.Println(s)

}

In eachiterat ionofthe loop, range produces a pair ofvalues: the index andthe value ofthe

elementatthatindex. Inthisexample, wedon’tneed the index, but the syntaxofarange lo op

re quires thatifwedealwit h theelement, wemustdealwit h theindex too.One ide a wouldbe

to assig n theindex to an obv iou sly temporar y var iable like temp andignoreits value,but Go

do es notper mitunu sed local variables, sothiswou ldresult in a compi lat ionerror.

Thesolut ion istouse the bl ank ident iﬁer,whose nameis _ (t hat is, an underscore). Theblank

identiﬁer may beusedwhene ver synt axrequires a variablenamebut prog ram log ic do es not,

forins tance todis cardanunwante d lo opindex whenwerequireonlythe elementvalue.Most

Go programmerswou ldlikelyuse range and _ to write the echo prog ram as abov e,since the

indexingover os.Args is imp licit, not explicit, andthu s easier toget rig ht.

Thisversionofthe program usesa short var iable decl arat iontodeclare and initialize s and

sep,but wecou ldequ allywel l have declare d thevar iables sep arately. There are several ways

to declare a str ing var iable; these are all equivalent:

s:=""

var s string

var s = ""

var s string = ""

Whyshouldyou preferone for m to another? The ﬁrs t form,ashort var iable decl arat ion, is

themostcompact,but it may beusedonlywit hin a function, not for package-le vel var iables.

Thesecon d form relies ondefau ltinitializat iontothe zerovalue for str ings, whichis "".The

thirdfor m is rarelyusedexceptwhendeclaring multiplevar iables. Thefourthfor m is explicit

ab out the var iable’styp e,whichisredundant whenitisthe sameasthatofthe initial value but

ne cessary inother cas es wherethe y arenot ofthe sametyp e.Inprac tice,you shouldgeneral ly

us e on e of the ﬁrs t twofor ms, wit h explicitinitializat iontosay thatthe initial value is

importantand imp licitinitializat iontosay thatthe initial value doesn’tmatter.

As not edabove , each timearo und the loop, the str ing s gets comp letelynew contents. The +=

st atement makes a new str ing bycon catenat ingthe old str ing , aspace charac ter,and the next

argument, thenassig nsthe newstr ing to s.The old cont entsof s arenolon g erinuse,sothe y

wi l l be garb age-collec ted indue course.

If the amountofdat a invo l vedislarge , this cou ldbecostly. A simpler andmoreefﬁcient

solut ion wou ldbetouse the Join func tionfro m the strings package:

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

8CHAPTER 1. TUTORIAL

gopl.io/ch1/echo3

func main() {

fmt.Println(strings.Join(os.Args[1:], " "))

}

Final ly, ifwedon’tcareabout for mat but justwanttosee the values, perhaps for debug ging, we

canlet Println format the results for us:

fmt.Println(os.Args[1:])

Theout put of thisstatement islikewhatwewou ldget fro m strings.Join,but wit h sur-

ro undingbrackets. Any slice may beprint edthisway.

Exercis e 1.1: Mo dif y the echo prog ram to als o pr int os.Args[0],the nameofthe command

that invo ked it.

Exercis e 1.2: Mo dif y the echo prog ram to print the index andvalue ofeachofits arguments,

on e perline.

Exercis e 1.3: Experiment tomeasure the dif ference in runningtimebet weenour pot ent ial ly

inefﬁcientversions and the one thatuses strings.Join.(Section1.6 illustrates par t of the

time package, and Sec tion 11.4 shows how towrite benchmark tests for systematicper-

formance evaluation.)

1.3. Finding DuplicateLines

Prog ramsfor ﬁle copy ing , pr int ing , searching, sor ting, count ing , andthe likeall haveasimi lar

st ruc ture: a loop overthe inp ut, som e comp utation on eachelement, andgenerat ionofout put

on the ﬂyoratthe end.We’ll showthree var iants ofaprogram cal le d dup;itispar tly ins pired

by the Unix uniq command, whichlooks for adj acentdup lic atelines. Thestr uctures and

packages usedare model s that can beeasi lyadapt ed.

The ﬁrs t versionof dup pr intseachlinethatapp earsmorethanonceinthe stand ard inp ut,

preceded byits count. Thisprogram introduces the if st atement,the map data typ e,and the

bufio package.

gopl.io/ch1/dup1

// Dup1 prints the text of each line that appears more than

// once in the standard input, preceded by its count.

package main

import (

"bufio"

"fmt"

"os"

)

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

SECTION 1.3. FINDING DUPLICATE LINES 9

func main() {

counts := make(map[string]int)

input := bufio.NewScanner(os.Stdin)

for input.Scan() {

counts[input.Text()]++

}

// NOTE: ignoring potential errors from input.Err()

for line, n := range counts {

if n > 1 {

fmt.Printf("%d\t%s\n", n, line)

}

As wit h for,parenthes es arenever usedaro und the con dit ion inan if st atement,but braces

arerequired for the body. There can beanopt ion al else part thatisexe cut edifthe con dit ion

is false.

A map holdsaset ofkey/value pairsand provides cons tant-t imeoperat ions tostore , retr ieve,

or testfor aniteminthe set. Thekey may beofany typ e whos e values can comp are d with ==,

st rings beingthe most commonexample; the value may beofany typ e at all. Inthisexample,

thekeysare str ingsand the values are ints. Thebui lt-in function make createsanew emp ty

map; ithas other usestoo.Maps aredis cussed at lengt h in Sec tion 4.3.

Each time dup re ads a lineofinp ut, the lineisusedasakey int o themap andthe cor-

resp ondingvalue isincrement ed. The statement counts[input.Text()]++ is equivalentto

thes e twostatements:

line := input.Text()

counts[line] = counts[line] + 1

It’snot a pro blem if the map doesn’tyet cont ain thatkey.The ﬁrs t time a newlineisseen, the

expression counts[line] on the rig ht-handside evaluatestothe zerovalue for itstyp e,which

is 0 for int.

To print the results, weuse another range-b ased for lo op, thistimeoverthe counts map. As

before , each iterat ionpro duces two results, a key and the value ofthe map elementfor that

ke y.The order ofmap iterat ionisnot speciﬁed,but inprac tice itisrandom, varying fro m on e

runtoanother.Thisdesig n is int ent ion al, since itpre vents programsfro m re lying onany par-

ticularorder ingwhere non e is guarante e d.

Onwardtothe bufio package, whichhelps makeinp utand out put efﬁcientand convenient.

Oneofits mostusefulfeaturesisatyp e called Scanner that reads inp utand bre aks itint o lines

or words;it’soften the easiestway topro cessinp utthatcom esnatural ly in lines.

Theprogram usesashort var iable decl arat iontocre ate a new var iable input that referstoa

bufio.Scanner:

input := bufio.NewScanner(os.Stdin)

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

10 CHAPTER 1. TUTORIAL

Thescanner reads fro m theprogram’sstand ard inp ut. Eachcal l to input.Scan() re ads the

next lineand remov esthe newlinecharac ter fro m theend;the resultcan beret rie ved bycal l-

ing input.Text().The Scan func tionretur ns true if there isalineand false when there is

no more inp ut.

Thefunction fmt.Printf,like printf in C andother langu ages, pro duces for matte d output

from a listofexpressions.Its ﬁrs t argumentisafor mat str ing thatspeciﬁes how subsequent

arguments shouldbefor matte d. Thefor mat ofeachargumentisdeter mined byaconversion

ch arac ter,aletterfol low ing a percentsig n. Fo r example, %d formats an integeroperandusing

de cimal not ation,and %s exp ands tothe value ofastr ing operand.

Printf hasoveradozen suchconversions,whichGoprogrammerscal l verb s.Thistable isfar

from a completespeciﬁcation but illustrates manyofthe featuresthatare avai lable:

%d de cimal integer

%x, %o, %b integer in hexade cimal,octal,binar y

%f, %g, %e ﬂoating-p ointnumber: 3.141593 3.141592653589793 3.141593e+00

%t boole an: true or false

%c rune (Unico de co de point)

%s st ring

%q quot edstr ing "abc" or rune 'c'

%v anyvalue in a natural for mat

%T type ofany value

%% literal percentsig n (nooperand)

Thefor mat str ing in dup1 also containsatab \t andanewline \n.Str ing lit eralsmay cont ain

such es cap e sequ ences forrepresent ing other wis e invisiblecharac ters. Printf do es notwrite a

ne wlinebydefau lt. Byconvent ion,for matting functions whose names endin f,suchas

log.Printf and fmt.Errorf,use the for matting rules of fmt.Printf,where asthose whose

namesend in ln fo llow Println,for matting their arguments as if by %v,fol low edbya

ne wline.

Many programsreadeit her fro m their stand ard inp ut, as abov e,orfro m asequence ofnamed

ﬁles. Thenext versionof dup canreadfro m thestand ard inp utorhandlealistofﬁle names,

using os.Open to openeachone:

gopl.io/ch1/dup2

// Dup2 prints the count and text of lines that appear more than once

// in the input. It reads from stdin or from a list of named files.

package main

import (

"bufio"

"fmt"

"os"

)

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

SECTION 1.3. FINDING DUPLICATE LINES 11

func main() {

counts := make(map[string]int)

files := os.Args[1:]

if len(files) == 0 {

countLines(os.Stdin, counts)

}else {

for _, arg := range files {

f, err := os.Open(arg)

if err != nil {

fmt.Fprintf(os.Stderr, "dup2: %v\n", err)

continue

}

countLines(f, counts)

f.Close()

}

for line, n := range counts {

if n > 1 {

fmt.Printf("%d\t%s\n", n, line)

}

func countLines(f *os.File, counts map[string]int) {

input := bufio.NewScanner(f)

for input.Scan() {

counts[input.Text()]++

}

// NOTE: ignoring potential errors from input.Err()

}

Thefunction os.Open returnstwo values. The ﬁrs t is anopenﬁle (*os.File)thatisusedin

su bsequentreads bythe Scanner.

Thesecon d resu ltof os.Open is a value ofthe bui lt-in error type.If err equals the speci al

built-in value nil,the ﬁle was opene d successf ully. The ﬁle isread, and whenthe end ofthe

inputisreach ed, Close clos es theﬁle andreleasesany res ources. On theother hand, if err is

not nil,som ethingwentwro ng. Inthatcas e,the error value descr ibesthe pro blem. Our sim-

ple-minde d er ror handlingprintsamessage onthe stand ard error streamusing Fprintf and

theverb %v,whichdispl ays a value ofany typ e in a default for mat, and dup then car r ies on

with thenext ﬁle; the continue st atement goestothe next iterat ionofthe enclosing for lo op.

In the int erestsofkeeping codesamples toareasonablesize,our early examples areint ent ion-

al lysom ewhat cavalier abouterror handling. Cle arly wemustche ckfor anerror fro m

os.Open;how ever, weare ignor ing the lesslikelypossibi lit y that an erro r couldocc ur while

re adingthe ﬁle wit h input.Scan.Wewill not e pl aces where we’ve skipp ederror che cking ,

andwewill goint o thedet ails oferror handling in Sec tion 5.4.

No tice thatthe cal l to countLines precedes its decl arat ion. Functions and other package-le vel

entities may bedeclare d in anyorder.

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

12 CHAPTER 1. TUTORIAL

Amap isareference to the dat a st ruc turecre ate d by make.Whenamap ispassedtoafunc-

tion,the functionreceivesa copyofthe reference,soany changesthe cal le d func tionmakes to

theunderly ing dat a st ruc turewill bevisible through the cal ler’smap reference too.Inour

example, the values inserted int o the counts mapby countLines areseenby main.

Theversions of dup ab ove operateina‘‘st reaming’’ mo deinwhichinp utisreadand bro ken

into lines as needed,soinprincip lethese programscan handleanarbit rar y amount ofinp ut.

An alt ernat ive appro ach istoreadthe ent ire inp utint o memory inone big gulp,split it int o

lines allatonce, thenpro cessthe lines. Thefol low ing version, dup3,operates in thatfashion.

It int roduces the function ReadFile (f rom the io/ioutil package), whichreads the ent ire

cont entsofaname d ﬁle,and strings.Split,whichsplitsastr ing int o aslice ofsubst rings.

(Split is the opp osite of strings.Join,whichwesaw earlier.)

We’ve simpliﬁed dup3 some what. First,itonlyreads name d ﬁles, not the stand ard inp ut, since

ReadFile re quires a ﬁle nameargument. Secon d,wemov edthe count ing ofthe lines back

into main,since itisnow needed inonlyone place.

gopl.io/ch1/dup3

package main

import (

"fmt"

"io/ioutil"

"os"

"strings"

)

func main() {

counts := make(map[string]int)

for _, filename := range os.Args[1:] {

data, err := ioutil.ReadFile(filename)

if err != nil {

fmt.Fprintf(os.Stderr, "dup3: %v\n", err)

continue

}

for _, line := range strings.Split(string(data), "\n") {

counts[line]++

}

for line, n := range counts {

if n > 1 {

fmt.Printf("%d\t%s\n", n, line)

}

ReadFile returnsabyteslice thatmustbeconverted int o a string so itcan besplit by

strings.Split.Wewill dis cussstr ingsand byteslices at lengt h in Sec tion 3.5.4.

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

SECTION 1.4. ANIMATED GIFS13

Un der the cov ers, bufio.Scanner, ioutil.ReadFile,and ioutil.WriteFile us e the Read

and Write methodsof *os.File,but it’srarethatmostprogrammersneed toaccessthose

lower-le vel routinesdirec tly.The hig her-le vel functions likethose fro m bufio and io/ioutil

areeasier touse.

Exercis e 1.4: Mo dif y dup2 to print the names ofall ﬁles in whicheachdup lic ated lineocc urs.

1.4. Animated GIFs

Thenext program demon strates basic usage ofGo’sstand ard image packages, whichwe’ll use

to cre ate a sequence ofbit-mappedimagesand thenencodethe sequence as a GIF animat ion.

Theimages, cal le d Li ssajous ﬁgure s,wereastaplevisualeffec t in sci-ﬁﬁlmsofthe 1960s. They

arethe paramet ric cur ves pro duce d by har monic osci l lat ionintwo dimensions,suchastwo

sinewaves fed int o the x and y inputs of an oscillos cope.Figure1.1 shows som e examples.

Figure 1.1. Fo urLissajous ﬁgures.

Thereare several new con str uctsinthiscode, including const de clarat ions,str uct typ es, and

comp osite lit erals. Unlikemostofour examples, thisone als o invo l vesﬂoating-p ointcom-

putation s.We’ll dis cussthese topics onlybrieﬂyhere, pushingmostdet ails off tolater chap-

ters,since the primary goalrig htnow istogiveyou anide a of whatGolooks like and the

kind s of things thatcan bedon e easi lywit h thelangu ageand its librar ies.

gopl.io/ch1/lissajous

// Lissajous generates GIF animations of random Lissajous figures.

package main

import (

"image"

"image/color"

"image/gif"

"io"

"math"

"math/rand"

"os"

)

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

14 CHAPTER 1. TUTORIAL

var palette = []color.Color{color.White, color.Black}

const (

whiteIndex = 0 // first color in palette

blackIndex = 1 // next color in palette

)

func main() {

lissajous(os.Stdout)

}

func lissajous(out io.Writer) {

const (

cycles = 5//number of complete x oscillator revolutions

res = 0.001 // angular resolution

size = 100 // image canvas covers [-size..+size]

nframes = 64 // number of animation frames

delay = 8//delay between frames in 10ms units

)

freq := rand.Float64() * 3.0 // relative frequency of y oscillator

anim := gif.GIF{LoopCount: nframes}

phase := 0.0 // phase difference

for i := 0; i < nframes; i++ {

rect := image.Rect(0, 0, 2*size+1, 2*size+1)

img := image.NewPaletted(rect, palette)

for t := 0.0; t < cycles*2*math.Pi; t += res {

x:=math.Sin(t)

y:=math.Sin(t*freq + phase)

img.SetColorIndex(size+int(x*size+0.5), size+int(y*size+0.5),

blackIndex)

}

phase += 0.1

anim.Delay = append(anim.Delay, delay)

anim.Image = append(anim.Image, img)

}

gif.EncodeAll(out, &anim) // NOTE: ignoring encoding errors

}

Af ter importingapackagewhose pat h hasmultiplecomponents, like image/color,werefer

to the packagewit h anamethatcom esfro m thelastcomponent.Thu s thevar iable

color.White belongstothe image/color packageand gif.GIF belongsto image/gif.

A const de clarat ion(§3.6) gives names tocon stants, thatis, values thatare ﬁxe d at comp ile

time,suchasthe numer ical parametersfor cyc les, frames, anddel ay. Like var de clarat ions,

const de clarat ions may appear at packagelevel (so the names arevisible throughoutthe pack-

age) orwit hin a function(so the names arevisible onlywit hin thatfunction). Thevalue ofa

cons tantmustbeanumber, str ing , or boole an.

Theexpressions []color.Color{...} and gif.GIF{...} are comp osite lit era ls (§4.2, §4.4.1),

acompact not ation for ins tantiat ingany ofGo’scomposite typ es from a sequence ofelement

values. Here, the ﬁrs t on e is a slice andthe secon d on e is a st ruc t.

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

SECTION 1.5. FETCHING A URL 15

Thetyp e gif.GIF is a str uct typ e (§4.4). A st ruc t is a gro upofvalues cal le d ﬁel ds,often ofdif-

ferent typ es, that arecol lec ted toget her in a singleobj e ctthatcan betre ate d as a unit. The

var iable anim is a str uct oftyp e gif.GIF.The str uct lit eral createsastr uct value whose Loop-

Count ﬁeldisset to nframes;all other ﬁelds have the zerovalue for their typ e.The indiv idu al

ﬁelds ofastr uct can beaccessedusingdot not ation,asinthe ﬁnaltwo assig nments which

explicitlyupdatethe Delay and Image ﬁelds of anim.

The lissajous func tionhas two neste d lo ops. Theout erlooprunsfor 64iterat ions,each

producingasingleframeofthe animat ion. Itcre atesanew 201&201 imagewit h apalette of

twocolor s,white and black.All pixel s areinitial lyset tothe palette’szerovalue (the zerot h

colorinthe palette), whichweset towhite. Eachpassthrough the inner loop generates a new

imagebysetting som e pixe lstoblack.The resultisapp ended,usingthe bui lt-in append func-

tion (§4.2.1), toalistofframes in anim,along wit h aspeciﬁed del ayof80ms. Final lythe

sequence offrames anddel ays isencoded int o GIF for mat andwritt entothe out put stream

out.The typ e of out is io.Writer,whichlets uswrite to a wide range ofpossibledestina-

tion s, as we’ll showsoon.

Theinner loop runsthe two oscillator s.The x os cillator isjustthe sinefunction. The y os cil-

latorisals o asinus oid,but itsfre quency rel ative tothe x os cillator isarandomnumber

between0and 3,and its phase rel ative tothe x os cillator isinitial lyzerobut incre aseswit h

each frameofthe animat ion. Thelooprunsunt i l the x os cillator has comp leted ﬁve full

cycles. Ateachstep, itcal ls SetColorIndex to color the pixel cor respondingto(x, y)black,

which is at position 1 inthe palette.

The main func tioncal lsthe lissajous func tion, direc ting ittowrite to the stand ard out put,

so thiscommand pro duces an animated GIF wit h frames like those inFigure1.1:

$gobuild gopl.io/ch1/lissajous

$./lissajous >out.gif

Exercis e 1.5: Change the Lissajous program’scolor palette to gre enonblack,for adde d

authenticity.Tocre ate the web color #

RRGGBB

,use color.RGBA{0x

,0x

,0xff},

whereeachpair ofhexade cimal digitsrepresentsthe int ensit y of the red,gre en, orbluecom-

ponent ofthe pixel.

Exercis e 1.6: Mo dif y theLissajous program topro duce images in multiplecolor s by adding

more values to palette andthendispl aying thembychang ing the thirdargumentof Set-

ColorIndex in som e interest ing way.

1.5. Fet ching a URL

Fo r many app lic ations,accesstoinfor mat ionfro m theInt ernet isasimp ortantasaccesstothe

lo cal ﬁle system. Goprovides a collec tion of packages, gro upedunder net,thatmakeiteasy

to sendand receive infor mat ionthrough the Int ernet, makelow-le vel networ k connec tion s,

andset upser vers, for whichGo’scon cur rency features(introduce d in Chapt er8)are par tic u-

larlyuseful.

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

16 CHAPTER 1. TUTORIAL

To illustratethe minimum necessary toret rie veinfor mat ionoverHTTP, h ere’sasimple

prog ram called fetch that fetch esthe content ofeachspeciﬁed URL andprintsitasuninter-

preted text; it’sins pired bythe invaluableutilit y curl.Obv iou sly one wou ldusu allydomore

with suchdat a, butthisshows the basic idea. Wewill use thisprogram fre quentlyinthe book.

gopl.io/ch1/fetch

// Fetch prints the content found at a URL.

package main

import (

"fmt"

"io/ioutil"

"net/http"

"os"

)

func main() {

for _, url := range os.Args[1:] {

resp, err := http.Get(url)

if err != nil {

fmt.Fprintf(os.Stderr, "fetch: %v\n", err)

os.Exit(1)

}

b, err := ioutil.ReadAll(resp.Body)

resp.Body.Close()

if err != nil {

fmt.Fprintf(os.Stderr, "fetch: reading %s: %v\n", url, err)

os.Exit(1)

}

fmt.Printf("%s", b)

}

Thisprogram introduces functions fro m twopackages, net/http and io/ioutil.The

http.Get func tionmakes an HTTP requestand,ifthere isnoerror,retur nsthe resultinthe

resp ons e st ruc t resp.The Body ﬁeldof resp cont ainsthe ser ver respons e as a readable

st ream. Next, ioutil.ReadAll re ads the ent ire resp ons e;the resultisstore d in b.The Body

st reamisclos edtoavoid leak ingres ources, and Printf wr itesthe respons e to the stand ard

output.

$gobuild gopl.io/ch1/fetch

$./fetch http://gopl.io

<html>

<head>

<title>The Go Programming Language</title>

...

If the HTTPrequestfai ls, fetch reportsthe fai lureins tead:

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

SECTION 1.6. FETCHING URLS CONCURRENTLY17

$./fetch http://bad.gopl.io

fetch: Get http://bad.gopl.io: dial tcp: lookup bad.gopl.io: no such host

In eit her erro r case, os.Exit(1) caus esthe pro cesstoexitwit h astatu s co de of 1.

Exercis e 1.7: Thefunctioncal l io.Copy(dst, src) re ads fro m src andwritesto dst.Use it

insteadof ioutil.ReadAll to copythe respons e bodyto os.Stdout withoutrequir inga

buffer large enoug h to holdthe ent ire stream. Besuretoche ckthe error resultof io.Copy.

Exercis e 1.8: Mo dif y fetch to add the preﬁx http:// to eachargumentURL if itismissing.

Yo u mig htwanttouse strings.HasPrefix.

Exercis e 1.9: Mo dif y fetch to als o pr int the HTTPstatu s co de,found in resp.Status.

1.6. Fet ching URLsConcurrently

Oneofthe most int erest ing and nove l aspectsofGoisits sup por t forcon cur rentprogram-

ming. Thisisalarge topic, towhichChapt er8 and Chapt er9 are devot ed, sofor now we’ll

give youjustataste ofGo’smain conc urrency mechanisms, goroutinesand channel s.

Thenext program, fetchall,doesthe samefetch ofaURL’scontentsasthe pre vious example,

butitfetch esmanyURLs, allcon cur rently, sothatthe pro cesswill takenolon g erthanthe

longest fetch rat her thanthe sum ofall the fetch times. Thisversionof fetchall discards the

resp ons esbut rep ortsthe size andelaps edtimefor eachone:

gopl.io/ch1/fetchall

// Fetchall fetches URLs in parallel and reports their times and sizes.

package main

import (

"fmt"

"io"

"io/ioutil"

"net/http"

"os"

"time"

)

func main() {

start := time.Now()

ch := make(chan string)

for _, url := range os.Args[1:] {

go fetch(url, ch) // start a goroutine

}

for range os.Args[1:] {

fmt.Println(<-ch) // receive from channel ch

}

fmt.Printf("%.2fs elapsed\n", time.Since(start).Seconds())

}

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

18 CHAPTER 1. TUTORIAL

func fetch(url string, ch chan<- string) {

start := time.Now()

resp, err := http.Get(url)

if err != nil {

ch <- fmt.Sprint(err) // send to channel ch

return

}

nbytes, err := io.Copy(ioutil.Discard, resp.Body)

resp.Body.Close() // don'tleak resources

if err != nil {

ch <- fmt.Sprintf("while reading %s: %v", url, err)

return

}

secs := time.Since(start).Seconds()

ch <- fmt.Sprintf("%.2fs %7d %s", secs, nbytes, url)

}

Here’s an example:

$gobuild gopl.io/ch1/fetchall

$./fetchall https://golang.org http://gopl.io https://godoc.org

0.14s 6852 https://godoc.org

0.16s 7261 https://golang.org

0.48s 2475 http://gopl.io

0.48s elapsed

A goro utine is a con cur rentfunctionexe cut ion.Ach ann el is a communic ationmechanism

that allows one goroutine topassvalues ofaspeciﬁed typ e to another goroutine.The function

main runs inagoroutine and the go st atement cre atesaddition algoroutines.

The main func tioncre atesachannel ofstr ingsusing make.For eachcommand-lineargument,

the go st atement inthe ﬁrs t range loopstartsanew goroutine thatcal ls fetch asy nchro nou sly

to fetch the URL using http.Get.The io.Copy func tionreads the bodyofthe respons e and

discards itbywriting tothe ioutil.Discard output stream. Copy returnsthe bytecount,

along wit h anyerror thatocc urre d.Aseachresultarrives, fetch send s asummar y lineonthe

ch annel ch.The secon d range loopin main re ceivesand printsthose lines.

Wh enone goroutine att emp tsasendorreceive onachannel,itblo cks until another goroutine

attemp tsthe cor respondingreceive orsendoperat ion, at whichpoint the value istransfer red

andbot h goro utinespro ceed.Inthisexample, each fetch send s avalue (ch <- expre ssi on)on

thechannel ch,and main re ceivesall ofthem(<-ch). Hav ing main do allthe print ing ens ures

that out put fro m each goroutine ispro cessedasaunit, wit h no dangerofint erleaving iftwo

goro utines ﬁnish atthe sametime.

Exercis e 1.10: Findaweb sit e that pro duces a large amountofdat a. Invest igatecachingby

running fetchall twice in successiontosee whether the rep orted time changesmuch. Do

youget the samecontent eachtime? Modif y fetchall to print its out put toaﬁle soitcan be

examined.

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

SECTION 1.7. AWEB SERVER 19

Exercis e 1.11: Tr y fetchall with lon g erargumentlists, suchassamples fro m thetop million

we b sitesavai lable at alexa.com.How doesthe program beh ave ifawe b site justdoesn’t

resp ond?(Section8.9 descr ibesmechanismsfor coping in suchcas es.)

1.7. A WebServer

Go’slibrar ies makesiteasy towrite a web ser ver thatrespond s to clientrequests like those

made by fetch.Inthissec tion,we’ ll showaminimal ser ver thatretur nsthe pat h comp onent

of the URL usedtoaccessthe ser ver.Thatis, if the requestisfor http://local-

host:8000/hello,the respons e wi l l be URL.Path = "/hello".

gopl.io/ch1/server1

// Server1 is a minimal "echo" server.

package main

import (

"fmt"

"log"

"net/http"

)

func main() {

http.HandleFunc("/", handler) // each request calls handler

log.Fatal(http.ListenAndServe("localhost:8000", nil))

}

// handler echoes the Path component of the request URL r.

func handler(w http.ResponseWriter, r *http.Request) {

fmt.Fprintf(w, "URL.Path = %q\n", r.URL.Path)

}

Theprogram isonlyahandf uloflines long because librar y func tions domostofthe wor k.

The main func tionconne cts a handler functiontoincomingURLs thatbeg in with /,whichis

al l URLs, andstartsaser ver listeningfor incomingrequests onpor t 8000. A re questisrep-

resent edasastr uct oftyp e http.Request,whichcontainsanumberofrel ate d ﬁelds,one of

whichisthe URL ofthe incomingrequest. Whenarequestarrives, itisgiven tothe handler

func tion, whichext racts the pat h comp onent (/hello)fro m therequestURL andsends it

back asthe respons e,using fmt.Fprintf.Web ser verswill beexplained indet ailin

Section7.7.

Let’sstart the ser ver in the backg round.OnMac OS X orLinux, add an ampers and (&)tothe

command; onMicros oft Windows, you will need torun the command wit houtthe amp ers and

in a sep aratecommand window.

$gorun src/gopl.io/ch1/server1/main.go &

We can thenmakeclientrequests fro m thecommand line:

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

20 CHAPTER 1. TUTORIAL

$gobuild gopl.io/ch1/fetch

$./fetch http://localhost:8000

URL.Path = "/"

$./fetch http://localhost:8000/help

URL.Path = "/help"

Alternat ive ly, wecan accessthe ser ver fro m aweb brows er, as shown inFigure1.2.

Figure 1.2. Arespons e from the ech o server.

It’seasy toadd featurestothe ser ver.One usefuladdition isaspeciﬁcURL thatretur nsa

st atu s of som e sort.For example, thisversiondoesthe sameech o butals o counts the number

of requests; a requesttothe URL /count returnsthe count sofar,exc luding /count re quests

themselves:

gopl.io/ch1/server2

// Server2 is a minimal "echo" and counter server.

package main

import (

"fmt"

"log"

"net/http"

"sync"

)

var mu sync.Mutex

var count int

func main() {

http.HandleFunc("/", handler)

http.HandleFunc("/count", counter)

log.Fatal(http.ListenAndServe("localhost:8000", nil))

}

// handler echoes the Path component of the requested URL.

func handler(w http.ResponseWriter, r *http.Request) {

mu.Lock()

count++

mu.Unlock()

fmt.Fprintf(w, "URL.Path = %q\n", r.URL.Path)

}

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

SECTION 1.7. AWEB SERVER 21

// counter echoes the number of calls so far.

func counter(w http.ResponseWriter, r *http.Request) {

mu.Lock()

fmt.Fprintf(w, "Count %d\n", count)

mu.Unlock()

}

Theser ver has two handlers, andthe requestURL deter mines whichone iscal le d: arequest

for /count invo kes counter andall othersinv oke handler.Ahand ler pattern that ends wit h

aslash match esany URL thathas the pattern asapreﬁx. Behindthe scenes, the ser ver runs

thehandler for eachincomingrequestinasep arategoroutine sothatitcan ser vemultiple

re quests simultane ously.How ever, iftwo con cur rentrequests try toupdate count at the same

time,itmig htnot beincrement edcon sistent ly; the program wou ldhaveaser ious bug cal le d a

ra c e condit ion (§9.1). Toavoid thispro blem, wemustens ure that at mostone goroutine

accessesthe var iable at a time, whichisthe pur pos e of the mu.Lock() and mu.Unlock() calls

that bracketeachaccessof count.We’ll lookmoreclos ely atcon cur rency wit h sh are d var i-

ables in Chapt er9.

As a richerexample, the handler functioncan rep ort onthe headersand for m data that it

re ceives, mak ingthe ser ver usefulfor ins pec ting and debug gingrequests:

gopl.io/ch1/server3

// handler echoes the HTTP request.

func handler(w http.ResponseWriter, r *http.Request) {

fmt.Fprintf(w, "%s %s %s\n", r.Method, r.URL, r.Proto)

for k, v := range r.Header {

fmt.Fprintf(w, "Header[%q] = %q\n", k, v)

}

fmt.Fprintf(w, "Host = %q\n", r.Host)

fmt.Fprintf(w, "RemoteAddr = %q\n", r.RemoteAddr)

if err := r.ParseForm(); err != nil {

log.Print(err)

}

for k, v := range r.Form {

fmt.Fprintf(w, "Form[%q] = %q\n", k, v)

}

Thisusesthe ﬁelds ofthe http.Request st ruc t to pro duce out put likethis:

GET /?q=query HTTP/1.1

Header["Accept-Encoding"] = ["gzip, deflate, sdch"]

Header["Accept-Language"] = ["en-US,en;q=0.8"]

Header["Connection"] = ["keep-alive"]

Header["Accept"] = ["text/html,application/xhtml+xml,application/xml;..."]

Header["User-Agent"] = ["Mozilla/5.0 (Macintosh; Intel Mac OS X 10_7_5)..."]

Host = "localhost:8000"

RemoteAddr = "127.0.0.1:59911"

Form["q"] = ["query"]

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

22 CHAPTER 1. TUTORIAL

No tice how the cal l to ParseForm is neste d within an if st atement.Goallowsasimplestate-

ment suchasalocal variabledeclarat iontopre cedethe if condit ion,whichispar tic ularly

us efulfor error handling as in thisexample. Wecou ldhavewritt enitas

err := r.ParseForm()

if err != nil {

log.Print(err)

}

butcom biningthe statementsisshorter andreduces the scope ofthe var iable err,whichis

go o d prac tice.We’ll deﬁne scope inSec tion 2.7.

In these programs, we’ve seenthree ver y dif ferenttyp es us edasout put streams.The fetch

prog ram copied HTTPrespons e data to os.Stdout,aﬁle,asdid the lissajous prog ram.

The fetchall prog ram thre w therespons e away (whi le counting its lengt h) by copying itto

thetrivialsin k ioutil.Discard.And the web ser ver abov e us ed fmt.Fprintf to write to an

http.ResponseWriter repres enting the web brows er.

Although these three typ es dif fer in the det ails ofwhatthe y do,the y al l satisf y acommon

interface,allow ing any ofthemtobeusedwhere ver an out put streamisneeded.Thatint er-

face,cal le d io.Writer,isdis cussed in Sec tion 7.1.

Go’sint erface mechanism isthe topic ofChapt er7,but togiveanide a of whatit’scap able of,

let’ssee how easy itistocom bine the web ser ver wit h the lissajous func tionsothatani-

mate d GIFs arewritt ennot tothe stand ard out put,but tothe HTTPclient. Justadd these

lines tothe web ser ver :

handler := func(w http.ResponseWriter, r *http.Request) {

lissajous(w)

}

http.HandleFunc("/", handler)

or equivalently:

http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {

lissajous(w)

})

Thesecon d argumenttothe HandleFunc func tioncal l immediate lyabove isafunc tionlit era l,

that is, an anonymou s func tiondeﬁne d at its point ofuse.Wewill explain itfur therin

Section5.6.

Once you’ve made thischange , visit http://localhost:8000 in yourbrows er.Eachtimeyou

lo ad thepage, you’ll see a new animat ionlikethe one inFigure1.3.

Exercis e 1.12: Mo dif y theLissajous ser ver toreadparameter values fro m theURL. For exam-

ple, you mig htarrange itsothataURL like http://localhost:8000/?cycles=20 sets the

numb erofcyc les to 20 ins teadofthe defau lt5.Use the strconv.Atoi func tiontoconvert the

st ringparameter into anint eger. You can see its document ation wit h go doc strconv.Atoi.

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

SECTION 1.8. LOOSE ENDS 23

Figure 1.3. Animated Lissajous ﬁgures in a brows er.

1.8. Loose Ends

Thereisalot moretoGothanwe’ve cov ere d in thisquickint roduc tion. Hereare som e topics

we’ve barelytouch eduponoromitt edent ire ly, wit h ju stenoug h discussionthatthe y wi l l be

fami liar whenthe y make brief appearances beforethe full treatment.

Contro l ﬂow : We cov ere d thetwo fundament alcontrol-ﬂow statements, if and for,but not

the switch st atement,whichisamulti-way branch.Here’sasmall example:

switch coinflip() {

case "heads":

heads++

case "tails":

tails++

default:

fmt.Println("landed on edge!")

}

Theresultofcal ling coinflip is compare d to the value ofeachcas e.Cas es areevaluate d from

toptobot tom,sothe ﬁrs t matchingone isexe cut ed. The opt ion aldefau ltcas e match esifnon e

of the other cas es do es; it may beplace d anywhere . Casesdonot fal l thro ugh fro m on e to the

next as in C-like langu ages (thoug h thereisararelyused fallthrough st atement thatover-

rides thisbeh avior).

A switch do es notneed anoperand; itcan justlistthe cas es, each ofwhichisaboole an

expression:

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

24 CHAPTER 1. TUTORIAL

func Signum(x int) int {

switch {

case x > 0:

return +1

default:

return 0

case x < 0:

return -1

}

Thisfor m is cal le d a taglessswitch;it’sequivalentto switch true.

Like the for and if st atements, a switch mayinclude an opt ion alsimplestatement—ashort

var iable decl arat ion, an incrementorassig nmentstatement,orafunctioncal l—that can be

us edtoset a value beforeitisteste d.

The break and continue st atementsmodif y theﬂow ofcontrol . A break caus escontrol to

resume atthe next statement after the inner most for, switch,or select st atement (which

we’ll see later), andaswesaw in Sec tion 1.3, a continue caus esthe inner most for lo opto

st art its next iterat ion. Statementsmay belab ele d so that break and continue canrefer to

them, for ins tance tobre akout of several neste d lo ops at onceortostart the next iterat ionof

theout ermostloop. There isevenagoto st atement,thoug h it’sint ended for machine-gener-

ated code, not regu lar use byprogrammers.

Na med types: A type de clarat ionmakes itpossibletogiveaname toanexist ing typ e.Since

st ruc t typesare often long , they are nearlyalways named.Afami liar exampleisthe deﬁnition

of a Point type for a 2-D graphics system:

type Point struct {

X, Y int

}

var p Point

Type declarat ions and named typ es arecov ere d in Chapt er2.

Po int ers: Go provides point ers,thatis, values thatcontain the addressofavar iable.Insom e

languages, not ably C,point ers are relative lyuncon strained.Inother langu ages, point ers are

disguisedas ‘‘references,’’ andthere’snot muchthatcan bedon e with them exceptpassthem

around.Gotakes a position som ewhereinthe midd le.Point ers are explicitlyvisible.The &

op erator yieldsthe addressofavar iable,and the * op erator ret rie ves the var iable thatthe

pointerrefersto, but there isnopoint erarithmetic. We’ll explain point ers inSec tion 2.3.2.

Me thods and inte rfa ces: Amet hod isafunctionass oci ated wit h anamed typ e; Go isunu sual

in thatmet hodsmay beatt ach edtoalmostany named typ e.Met hodsare cov ere d in Chap-

ter6.Int erfaces areabstrac t typesthatlet ustre atdif ferentcon crete typ es in the sameway

basedonwhatmet hodsthe y have , nothow the y arerepresent edorimp lemente d. Interfaces

arethe subjec t of Chapt er7.

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

SECTION 1.8. LOOSE ENDS 25

Pa ckage s: Go com eswit h an extensive stand ard librar y of usefulpackages, andthe Gocom-

munity has create d andshare d many more. Programmingisoften moreabout usingexist ing

packages thanabout writing originalcodeofone’sown.Throughoutthe book, wewill point

outacoupleofdozen ofthe most imp ortantstand ard packages, but there are manymorewe

don’thavespace tomention,and wecannot provide anyt hingremot ely likeacompleterefer-

ence for any package.

Before you emb ark onany new program, it’sagood ide a to see ifpackages already exist that

mig hthelpyou get yourjob don e more easi ly. You can ﬁnd anindex ofthe stand ard librar y

packages at https://golang.org/pkg andthe packages cont ribut edbythe community at

https://godoc.org.The go doc to olmakes these documents easi lyaccessiblefro m the

commandline:

$godoc http.ListenAndServe

package http // import "net/http"

func ListenAndServe(addr string, handler Handler) error

ListenAndServe listens on the TCP network address addr and then

calls Serve with handler to handle requests on incoming connections.

...

Comm ents: We havealready mention eddocumentation commentsatthe beg inningofa

prog ram or package. It’sals o go o d st yle towrite a comment beforethe declarat ionofeach

func tiontospecif y itsbeh avior.These convent ion s areimp ortant, because the y areusedby

to ols like go doc and godoc to locateand displ aydocumentation (§10.7.4).

Fo r comments thatspanmultiplelines orapp ear wit hin an expressionorstatement,there is

also the /* ... */ notation fami liar fro m ot her langu ages. Suchcommentsare som etimes

us edatthe beg inningofaﬁle for a large blo ckofexplanatory text toavoid a // on every line.

Wi thin a comment, // and /* have nospeci al me aning, socomments do not nest.

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

This page intentionally left blank

From the Library of YIGUANG HU

ptg16105617

ProgramStructure

In Go, asinany other programminglangu age, one bui ldslarge programsfro m asmall set of

basic cons tructs. Var iables store values. Simpleexpressions are com bine d into largerones

with operat ions likeaddition and subtrac tion. Basic typ es arecol lec ted int o ag gregates like

ar rays andstr ucts. Expressions are usedinstatementswhose exe cut ion order isdeter mined

by control-ﬂow statementslike if and for.Statementsare gro upedint o func tions for

is olation and reuse.Functions are gat hered int o source ﬁles andpackages.

We saw examples ofmostofthese inthe pre vious chapt er. Inthischapt er, we’ll goint o more

det ailabout the basic str uctural elements ofaGoprogram. Theexampleprogramsare int en-

tion allysimple, sowecan focus onthe langu agewit houtgetting sidet racke d by complic ated

algor it hms ordat a st ruc tures.

2.1. Names

Thenames ofGofunctions,var iables, cons tants, typ es, st atement lab els,and packages fol low a

simplerule: a namebeg inswit h aletter(that is, anyt hingthatUnico de de emsa letter) oran

underscoreand may haveany numberofaddition alletters,dig its, andunderscores. Cas e mat-

ters: heapSort and Heapsort aredif ferentnames.

Go has 25 ke yword s li ke if and switch that may beusedonlywhere the syntaxper mits; the y

can’tbeused as names.

break default func interface select

case defer go map struct

chan else goto package switch

const fallthrough if range type

continue for import return var

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

28 CHAPTER 2. PROGRAM STRUCTURE

In addition,there are about three dozen pred eclare d nameslike int and true forbui lt-in con-

st ants, typ es, andfunctions:

Cons tants: true false iota nil

Types: int int8 int16 int32 int64

uint uint8 uint16 uint32 uint64 uintptr

float32 float64 complex128 complex64

bool byte rune string error

Func tions: make len cap new append copy close delete

complex real imag

panic recover

Thes e namesare not res erve d,soyou may use themindeclarat ions.We’ll see a handf ulof

pl aces where rede claring one ofthemmakes sense,but bewareofthe pot ent ial for confusion.

If anent ity isdeclare d within a function, itis local to thatfunction. Ifdeclare d outside ofa

func tion, how ever, itisvisible in all ﬁles ofthe packagetowhichitbelon gs. Thecas e of the

ﬁrs t letterofanamedeter mines its visibilit y acrosspackagebound aries. Ifthe namebeg ins

with anupp er-case letter, itis expor ted,whichmeans thatitisvisible andaccessibleoutside of

itsown packageand may berefer red tobyother par ts of the program, as wit h Printf in the

fmt package. Packagenames thems elves arealways in lowercas e.

Thereisnolimitonnamelengt h, butconvent ion and sty leinGoprogramsleantowardshort

names, especi ally for local variables wit h smal l scop es; you are muchmorelikelytosee var i-

ables named i than theLoopIndex.General ly, the largerthe scope ofaname, the lon g erand

more meaningf ul it shouldbe.

St ylist ically, Goprogrammersuse ‘‘camel cas e’’ when for mingnames bycom biningwords;that

is,int erior capit alletters are preferred overint erior underscores. Thus the stand ard librar ies

have functions wit h nameslike QuoteRuneToASCII and parseRequestLine butnever

quote_rune_to_ASCII or parse_request_line.The letters ofacronymsand initialismslike

ASCII andHTMLare always rendered inthe samecas e,soafunctionmig htbecal le d html-

Escape, HTMLEscape,or escapeHTML,but not escapeHtml.

2.2. Declarations

A decl arati on namesaprogram entity and speciﬁes som e or all ofits pro per ties. Thereare

four maj orkinds ofdeclarat ions: var, const, type,and func.We’ll tal k ab out var iables and

types in thischapt er, con stants in Chapt er 3, and functions inChapt er5.

AGoprogram isstore d in one ormoreﬁles whose names endin .go.Each ﬁle beg inswit h a

package de clarat ionthatsays whatpackagethe ﬁle ispar t of . The package de clarat ionis

fo llowe d by any import de clarat ions,and thenasequence of pack age-level de clarat ions of

types, variables, cons tants, andfunctions,inany order.For example, thisprogram decl aresa

cons tant, a function, andacoupleofvar iables:

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

SECTION 2.2. DECLARATIONS 29

gopl.io/ch2/boiling

// Boiling prints the boiling point of water.

package main

import "fmt"

const boilingF = 212.0

func main() {

var f = boilingF

var c = (f - 32) * 5 / 9

fmt.Printf("boiling point = %g°F or %g°C\n", f, c)

// Output:

// boiling point = 212°F or 100°C

}

Thecon stant boilingF is a pac

kage-le vel declarat ion(as is main), where asthe var iables f and

c arelocal tothe function main.The nameofeachpackage-le vel ent ity isvisible not only

thro ughoutthe sourceﬁle thatcontainsits decl arat ion, but throughoutall the ﬁles ofthe pack-

age. Bycontrast, localdeclarat ions are visible onlywit hin the functioninwhichthe y are

de clare d andperhaps onlywit hin a smal l part ofit.

Afunctiondeclarat ionhas a name, a listofparameters(thevar iables whose values are

prov ide d by the function’scal lers), an opt ion allistofresults, andthe functionbody, which

cont ainsthe statementsthatdeﬁne whatthe functiondoes. Theresultlistisomitt edifthe

func tiondoesnot retur n anyt hing. Exe cut ion of the functionbeg inswit h theﬁrs t st atement

andcontinues until itencount ers a retur n st atement orreach esthe end ofafunctionthathas

no results. Control and any results arethenretur ned tothe cal ler.

We’ve seenafair numberoffunctions already and there are lotsmoretocom e,includingan

extensive dis cussioninChapt er5,sothisisonlyasketch.The function fToC belowenc apsu-

latesthe temperatureconversionlog ic so thatitisdeﬁne d on lyoncebut may beusedfro m

mu ltipleplaces. Here main callsittwice,usingthe values oftwo dif ferentlocal cons tants:

gopl.io/ch2/ftoc

// Ftoc prints two Fahrenheit-to-Celsius conversions.

package main

import "fmt"

func main() {

const freezingF, boilingF = 32.0, 212.0

fmt.Printf("%g°F = %g°C\n", freezingF, fToC(freezingF)) // "32°F = 0°C"

fmt.Printf("%g°F = %g°C\n", boilingF, fToC(boilingF)) // "212°F = 100°C"

}

func fToC(f float64) float64 {

return (f - 32) * 5 / 9

}

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

30 CHAPTER 2. PROGRAM STRUCTURE

2.3. Variables

A var de clarat ioncre atesavar iable ofapar tic ulartyp e,att ach esanametoit, andsets its ini-

tial value.Eachdeclarat ionhas the general for m

var

name type

expression

Either the typ e or the =

expression

part may beomitt ed, but not bot h.Ifthe typ e is omitt ed,

it isdeter mined bythe initializer expression. Ifthe expressionisomitt ed, the initial value is

the zero value forthe typ e,whichis 0 fornumbers, false forboole ans, "" forstr ings, and nil

forint erfaces andreference typ es (slice,point er, map,channel,function). Thezerovalue ofan

ag gregatetyp e li ke an array orastr uct has the zerovalue ofall ofits elements or ﬁelds.

Thezero-value mechanism ensuresthatavar iable always holds a wel l-deﬁne d value ofits typ e;

in Gothere isnosuchthingasanuninitialize d var iable.Thissimpliﬁes code andoften

ensuressensiblebeh avior of bound ary con dit ion s withoutext ra work.For example,

var s string

fmt.Println(s) // ""

pr intsanemp tystr ing , rat her thancausingsom e kind oferror or unp redic table beh avior.Go

prog rammersoften gotosom e ef for t to makethe zerovalue ofamorecomplic ated typ e

me aningf ul,sothatvar iables beg in life in a usefulstate.

It ispossibletodeclare and opt ion allyinitialize a set ofvar iables in a singledeclarat ion, wit h a

matchinglistofexpressions.Omitt ing the typ e al lows decl arat ionofmultiplevar iables ofdif-

ferent typ es:

var i, j, k int // int, int, int

var b, f, s = true, 2.3, "four" // bool, float64, string

Init ializersmay belit eral values orarbit rar y expressions.Package-le vel var iables areinitial-

ize d before main begins (§2.6.2), andlocal variables areinitialize d as their decl arat ions are

encountere d during functionexe cut ion.

Aset ofvar iables can also beinitialize d by cal lingafunctionthatretur nsmultiplevalues:

var f, err = os.Open(name) // os.Open returns a file and an error

2.3.1. Short Variable Declarations

Wi thin a function, an alternatefor m called a sh ort var iab ledeclarati on maybeusedtodeclare

andinitialize localvar iables. Ittakes the for m

name

expression

,and the typ e of

name

deter mined bythe typ e of

expression

.Hereare three ofthe manyshort var iable decl arat ions

in the lissajous func tion(§1.4):

anim := gif.GIF{LoopCount: nframes}

freq := rand.Float64() * 3.0

t:=0.0

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

SECTION 2.3. VARIABLES 31

Becaus e of their bre vit y and ﬂexibi lit y,short var iable decl arat ions are usedtodeclare and ini-

tialize the maj ority oflocal variables. A var de clarat iontends toberes erve d forlocal variables

that need anexplicittyp e that differs fro m that ofthe initializer expression, orfor whenthe

var iable will beassig ned a value later andits initial value isunimportant.

i:=100 // an int

var boiling float64 = 100 // a float64

var names []string

var err error

var p Point

As wit h var de clarat ions,multiplevar iables may bedeclare d andinitialize d in the sameshort

var iable decl arat ion,

i, j := 0, 1

butdeclarat ions wit h mu ltipleinitializer expressions shouldbeusedonlywhenthe y helpread-

abilit y,such as for short and natural gro upingslikethe initializat ionpar t of a for lo op.

Ke epinmindthat := is a declarat ion, where as = is anassig nment. A mu lti-var iable decl ara-

tion shouldnot beconfusedwit h a tupl e assig nment (§2.4.1), in whicheachvar iable onthe

lef t-handside isassig ned the cor respondingvalue fro m therig ht-handside:

i, j = j, i // swap values of i and j

Like ordinar y var de clarat ions,short var iable decl arat ions may beusedfor cal lstofunctions

li ke os.Open that retur n twoormorevalues:

f, err := os.Open(name)

if err != nil {

return err

}

// ...use f...

f.Close()

Onesubtlebut imp ortantpoint:ashort var iable decl arat iondoesnot necessarily decl are al l the

var iables onits left-handside.Ifsom e of themwerealready declare d in the same lexic al block

(§2.7), thenthe short var iable decl arat ionactslikean assig nment to those var iables.

In the codebelow,the ﬁrs t st atement declaresbot h in and err.The secon d de clares out but

on lyassig nsavalue tothe exist ing err var iable.

in, err := os.Open(infile)

// ...

out, err := os.Create(outfile)

Ashort var iable decl arat ionmustdeclare atleast one newvar iable,how ever, sothiscodewill

notcompi le:

f, err := os.Open(infile)

// ...

f, err := os.Create(outfile) // compile error: no new variables

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

32 CHAPTER 2. PROGRAM STRUCTURE

The ﬁxistouse anordinar y assig nmentfor the secon d st atement.

Ashort var iable decl arat ionactslikeanassig nmentonlytovar iables thatwerealready

de clare d in the samelexic al block; decl arat ions inanout erblo ckare ignored.We’ll see exam-

ples ofthis at the end ofthe chapt er.

2.3.2. Pointers

A variab le is a pie ce of storagecontainingavalue.Var iables create d by declarat ions are iden-

tiﬁed byaname, suchas x,but manyvar iables areidentiﬁed onlybyexpressions like x[i] or

x.f.All these expressions readthe value ofavar iable,exceptwhenthe y appear onthe lef t-

hand side of an assig nment, in whichcas e anew value isassig ned tothe var iable.

A pointer value isthe ad dre ss of a var iable.Apointeristhu s thelocat ionatwhichavalue is

stored.Not every value has an address, but every var iable does. Wit h apoint er, wecan read

or update the value ofavar iable in direc tly,wit houtusingorevenknowing the nameofthe

var iable, if indeed ithas a name.

If a var iable isdeclare d var x int,the expression &x (‘‘addressof x’’)yieldsapoint ertoan

integervar iable,thatis, a value oftyp e *int,whichispro nounced ‘‘pointertoint.’’ If this

value iscal le d p,wesay ‘‘p points to x,’’ or equivalently ‘‘p cont ainsthe addressof x.’’ Thevar i-

able towhich p points iswritt en *p.The expression *p yields the value ofthatvar iable,an

int,but since *p denot esavar iable,itmay also app ear onthe lef t-handside ofanassig nment,

in whichcas e theassig nmentupdates the var iable.

x:=1

p:=&x//p,oftype *int, points to x

fmt.Println(*p) // "1"

*p = 2 // equivalent to x = 2

fmt.Println(x) // "2"

Each component ofavar iable ofaggregatetyp e—a ﬁeldofastr uct oranelementofanarray—

is als o avar iable andthu s has an addresstoo.

Variables aresom etimesdes crib edas ad dre ssabl e values. Expressions thatdenot e var iables are

theonlyexpressions towhichthe ad dre ss-of op erator & maybeapp lie d.

Thezerovalue for a point erofany typ e is nil.The test p!=nil is trueif p points toavar i-

able.Point ers are comparable; two point ers are equ alifand onlyifthe y pointtothe same

var iable orbot h arenil.

var x, y int

fmt.Println(&x == &x, &x == &y, &x == nil) // "true false false"

It isper fec tly safefor a functiontoretur n theaddressofalocal variable. For ins tance,inthe

co de below, the local variable v create d by thispar tic ularcal l to f wi l l remain in existence even

af ter the cal l hasretur ned,and the point er p wi l l st i l l refertoit:

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

SECTION 2.3. VARIABLES 33

var p = f()

func f() *int {

v:=1

return &v

}

Each cal l of f returnsadistinc t value:

fmt.Println(f() == f()) // "false"

Becaus e apoint ercontainsthe addressofavar iable,passingapoint erargumenttoafunction

makesitpossiblefor the functiontoupdatethe var iable thatwas indirec tly passed. For exam-

ple, thisfunctionincrementsthe var iable thatits argumentpointstoand retur nsthe new

value ofthe var iable soitmay beused in an expression:

func incr(p *int) int {

*p++ // increments what p points to; does not change p

return *p

}

v:=1

incr(&v) // side effect: v is now 2

fmt.Println(incr(&v)) // "3" (and v is 3)

Each timewetakethe addressofavar iable orcopyapoint er, wecre ate new aliases or ways to

identify the samevar iable.For example, *p is anali as for v.Point erali asingisusefulbecause

it allowsustoaccessavar iable wit houtusingits name, but thisisadou ble-e dged sword:to

ﬁnd all the statementsthataccessavar iable,wehavetoknowall its alias es. It’snot justpoint-

ersthatcre ate ali ases; aliasingals o occurs whenwecopyvalues ofother reference typ es li ke

slices, maps, andchannel s,and evenstr ucts, arrays, andint erfaces thatcontain these typ es.

Po int ers are key tothe flag package, whichusesaprogram’scommand-linearguments toset

thevalues ofcer tain variables dist ribut edthroughoutthe program. Toillustrate, thisvar iat ion

on the earlier echo commandtakes two opt ion al ﬂags: -n caus es echo to omitthe trai ling

ne wlinethatwou ldnor mal lybeprint ed, and -s sep caus esittosep aratethe out put argu-

mentsbythe contentsofthe str ing sep insteadofthe defau ltsinglespace.Since thisisour

four thversion, the packageiscal le d gopl.io/ch2/echo4.

gopl.io/ch2/echo4

// Echo4 prints its command-line arguments.

package main

import (

"flag"

"fmt"

"strings"

)

var n = flag.Bool("n", false, "omit trailing newline")

var sep = flag.String("s", " ", "separator")

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

34 CHAPTER 2. PROGRAM STRUCTURE

func main() {

flag.Parse()

fmt.Print(strings.Join(flag.Args(), *sep))

if !*n {

fmt.Println()

}

Thefunction flag.Bool createsanew ﬂag variableoftyp e bool.Ittakes three arguments: the

name ofthe ﬂag ("n"), the var iable’sdefau ltvalue (false), andamessage thatwill beprint ed

if the userprovides an invalid argument, an invalid ﬂag , or -h or -help.Simi larly,

flag.String takesaname, a defau ltvalue,and a mess age , andcre atesastring var iable.The

var iables sep and n arepoint ers to the ﬂag variables, whichmustbeaccessedindirec tly as

*sep and *n.

Wh enthe program isrun, itmustcal l flag.Parse before the ﬂags areused, to updatethe ﬂag

var iables fro m their default values. Thenon-ﬂag arguments areavai lable fro m flag.Args()

as a slice ofstr ings. If flag.Parse encounters anerror,itprintsausage message and cal ls

os.Exit(2) to terminatethe program.

Let’srun som e test cas es on echo:

$gobuild gopl.io/ch2/echo4

$./echo4 a bc def

abcdef

$./echo4 -s / a bc def

a/bc/def

$./echo4 -n a bc def

abcdef$

$./echo4 -help

Usage of ./echo4:

-n omit trailing newline

-s string

separator (default " ")

2.3.3. The new Function

Anot her way tocre ate a var iable istouse the bui lt-in function new.The expression new(T)

createsan unname d variab le of typ e T,initializes ittothe zerovalue of T,and retur nsits

address, whichisavalue oftyp e *T.

p:=new(int) // p, of type *int, points to an unnamed int variable

fmt.Println(*p) // "0"

*p = 2 // sets the unnamed int to 2

fmt.Println(*p) // "2"

Avar iable create d with new is nodif ferentfro m an ordinar y lo cal variablewhose addressis

taken, exceptthatthere’snoneed toinv ent (anddeclare)adummyname, and wecan use

new(T) in an expression. Thus new is onlyasyntactic conv enience,not a fundament alnot ion:

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

SECTION 2.3. VARIABLES 35

thetwo newInt func tions below haveidenticalbeh avior s.

func newInt() *int { func newInt() *int {

return new(int) var dummy int

}return &dummy

}

Each cal l to new returnsadistinc t var iable wit h aunique address:

p:=new(int)

q:=new(int)

fmt.Println(p == q) // "false"

Thereisone exception tothisrule: two var iables whose typ e carries noinfor mat ionand is

thereforeofsize zero, suchas struct{} or [0]int,may,dep endingonthe imp lementation,

have the sameaddress.

The new func tionisrel ative lyrarelyusedbecause the most commonunnamed var iables areof

st ruc t types, for whichthe str uct lit eral synt ax(§4.4.1) ismore ﬂexible.

Since new is a pre declare d func tion, not a key word, it’spossibletoredeﬁne the namefor

somethingels e within a function, for example:

func delta(old, new int) int { return new - old }

Of course,wit hin delta,the bui lt-in new func tionisunavai lable.

2.3.4. Lifetime ofVariables

The li fetim e of a var iable isthe int erval oftimedur ingwhichitexistsasthe program exec utes.

Thelifet imeofapackage-le vel var iable isthe ent ire exe cut ion of the program. Bycontrast,

lo cal variables havedynamic lifet imes: a new ins tance iscre ate d each timethe declarat ion

st atement isexe cut ed, and the var iable livesonunt i l it becom es unre ach abl e,atwhichpoint its

storagemay berec ycled.Functionparametersand results arelocal variables too;the y arecre-

ated eachtimetheir enclosingfunctioniscal le d.

Fo r example, inthisexcer ptfro m theLissajous program ofSec tion 1.4,

for t := 0.0; t < cycles*2*math.Pi; t += res {

x:=math.Sin(t)

y:=math.Sin(t*freq + phase)

img.SetColorIndex(size+int(x*size+0.5), size+int(y*size+0.5),

blackIndex)

}

thevar iable t is cre ate d each timethe for lo opbeg ins, andnew var iables x and y arecre ate d

on eachiterat ionofthe loop.

Ho w do es thegarb agecol lec tor knowthatavar iable’sstoragecan bereclaimed?The full story

is muchmoredet aile d than weneed here, but the basic ideaisthatevery package-le vel var i-

able,and every local variableofeachcur rentlyactivefunction, can pot ent ial lybethe start or

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

36 CHAPTER 2. PROGRAM STRUCTURE

ro otofapat h to the var iable in question,fol low ing point ers and other kinds ofreferences that

ultimate lyleadtothe var iable.Ifnosuchpat h exists, the var iable has becom e unreach able, so

it can nolon g eraffec t therestofthe computation.

Becaus e thelifet imeofavar iable isdeter mined onlybywhether ornot itisreach able, a local

var iable may out liveasingleiterat ionofthe enclosingloop. Itmay cont inuetoexist evenafter

itsenclosingfunctionhas retur ned.

Acompi ler may cho ose toallocatelocal variables onthe heap oronthe stack but,perhaps sur-

pr isingly, thischoice isnot deter mined bywhether var or new was usedtodeclare the var iable.

var global *int

func f() { func g() {

var x int y:=new(int)

x=1 *y = 1

global = &x }

}

Here , x mu stbeheap-al locate d becaus e it isstill reach ablefro m thevar iable global af ter f

hasretur ned,despit e beingdeclare d as a localvar iable; wesay x es cap esfro m f.Convers ely,

when g returns, the var iable *y becomesunreach ableand can berec ycled.Since *y do es not

es cap e from g,it’ssafefor the compi ler toallocate *y on the stack,eventhoug h it was allo-

cate d with new.Inany cas e,the notion of escapingisnot som ethingthatyou need towor ry

ab out inorder towrite cor rec t co de,thoug h it’sgood tokeepinminddur ingper for mance

opt imizat ion, since eachvar iable thatescap es re quires an extra memor y al location.

Garb agecol lec tion isatremendou s helpinwriting cor rec t prog rams, but it doesnot relie ve

youofthe burden ofthin kingabout memor y.You don’tneed toexplicitlyallocateand fre e

memory,but towrite efﬁcientprogramsyou still need tobeawareofthe lifet imeofvar iables.

Fo r example, keeping unnecessary point ers to short-live d objec tswit hin long-live d objec ts,

especi ally globalvar iables, will pre ventthe garb agecol lec tor fro m re claimingthe short-live d

objec ts.

2.4. Assignments

Thevalue heldbyavar iable isupdated byanassig nmentstatement,whichinits simplestfor m

hasavar iable onthe lef t of the = sig n and an expressiononthe rig ht.

x=1 // named variable

*p = true // indirect variable

person.name = "bob" // struct field

count[x] = count[x] * scale // array or slice or map element

Each ofthe arithmeticand bit w ise binar y op erator s hasacor responding assig nment operator

al lowing , forexample, the laststatement toberewritt enas

count[x] *= scale

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

SECTION 2.4. ASSIGNMENTS37

whichsaves usfro m having torep eat (andre-e valuate)the expressionfor the var iable.

Numericvar iables can also beincrement edand decrement edby ++ and -- st atements:

v:=1

v++ // same as v = v + 1; v becomes 2

v-- // same as v = v - 1; v becomes 1 again

2.4.1. Tuple Assignment

Anot her for m of assig nment, known as tupl e assig nment,allowsseveral variables tobe

assig ned atonce. All ofthe rig ht-handside expressions are evaluate d before any ofthe var i-

ables areupdated,mak ingthisfor m most usefulwhensom e of the var iables appear onbot h

sides ofthe assig nment, as happens,for example, whenswapping the values oftwo var iables:

x, y = y, x

a[i], a[j] = a[j], a[i]

or whencomputing the gre atest commondiv isor(GCD) oftwo int egers:

func gcd(x, y int) int {

for y != 0 {

x, y = y, x%y

}

return x

}

or whencomputing the n-t h Fibonacci numberiterat ive ly:

func fib(n int) int {

x, y := 0, 1

for i := 0; i < n; i++ {

x, y = y, x+y

}

return x

}

Tu ple assig nmentcan also makeasequence oftrivialassig nments morecompact,

i, j, k = 2, 3, 5

though asamatterofsty le, avoid the tup lefor m if the expressions are complex; a sequence of

separatestatementsiseasier toread.

Certain expressions,suchasacal l to a functionwit h mu ltipleresults, pro duce several values.

Wh ensuchacal l is used inanassig nmentstatement,the lef t-handside musthaveasmany

var iables as the functionhas results.

f, err = os.Open("foo.txt") // function call returns two values

Of ten, functions use these addition alresults toindic atesom e kind oferror,eit her byretur ning

an error as in the cal l to os.Open,orabool,usu allycal le d ok.Aswe’ll see inlater chapt ers,

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

38 CHAPTER 2. PROGRAM STRUCTURE

thereare three operator s that som etimesbeh ave thisway too.Ifamap lookup (§4.3), typ e

assertion(§7.10), orchannel receive (§8.4.2) appearsinanassig nmentinwhichtwo results are

exp ected,eachpro duces an addition alboole an resu lt:

v, ok = m[key] // map lookup

v, ok = x.(T) // type assertion

v, ok = <-ch // channel receive

As wit h var iable decl arat ions,wecan assig n unwante d values tothe blank identiﬁer :

_, err = io.Copy(dst, src) // discard byte count

_, ok = x.(T) // check type but discard result

2.4.2. Assignability

As sig nmentstatementsare anexplicitfor m of assig nment, but there are manyplaces in a

prog ram whereanassig nmentocc urs implicit ly:a functioncal l implicitlyassig nsthe argument

values tothe cor respondingparameter variables; a return st atement imp licitlyassig nsthe

return op erands tothe cor respondingresultvar iables; andalit eral expressionfor a composite

type (§4.2) such as thisslice:

medals := []string{"gold", "silver", "bronze"}

implicitlyassig nseachelement, as if ithad beenwritt enlikethis:

medals[0] = "gold"

medals[1] = "silver"

medals[2] = "bronze"

Theelements ofmaps andchannel s,thoug h notordinar y var iables, areals o su bjec t to simi lar

implicitassig nments.

An assig nment, explicitorimp licit, isalways legal if the lef t-handside (the var iable) andthe

right-handside (the value) havethe sametyp e.Moregeneral ly, the assig nmentislegal onlyif

thevalue is assig nab le to the typ e of the var iable.

Therulefor assig nab ility hascas es forvar ious typ es, so we’ll explain the relevantcas e as we

introduce eachnew typ e.For the typ es we’ve dis cussedsofar,the rules aresimple: the typ es

mu stexac tly match,and nil maybeassig ned toany var iable ofint erface orreference typ e.

Cons tants (§3.6) havemore ﬂexiblerules for assig nabilit y that avoidthe need for mostexplicit

conv ersions.

Wh ether two values may becompare d with == and != is rel ate d to assig nabilit y:inany com-

parison, the ﬁrs t op erandmustbeassig nable tothe typ e of the secon d op erand, orvice versa.

As wit h assig nabilit y,we’ll explain the relevantcas es for comp arab ility when wepresent each

ne w type.

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

SECTION 2.5. TYPE DECLARATIONS 39

2.5. Type Declarations

Thetyp e of a var iable orexpressiondeﬁnesthe charac ter ist ics ofthe values itmay takeon,

such astheir size (numberofbitsornumberofelements, perhaps), how the y arerepresent ed

internal ly, the int rinsic operat ions thatcan beper for med onthem, andthe methodsass oci-

ated wit h them.

In any program there are var iables thatshare the samerepresent ation but sig nifyver y dif fer-

entcon cepts. For ins tance,an int couldbeusedtorepresent a loopindex, a timestamp,aﬁle

des crip tor,oramonth;afloat64 couldrepresent a velocit y in metersper secon d or a temper-

atureinone ofseveral scales; andastring couldrepresent a passwordorthe nameofacolor.

A type de clarat iondeﬁnesanew name d ty pe that has the same un derly ing type as an exist ing

type.The named typ e prov ides a way tosep aratedif ferentand perhaps incomp atibleusesof

theunderly ing typ e so thatthe y can’tbemixed unintent ion ally.

type

name underlying-type

Type declarat ions mostoften appear at packagelevel,where the named typ e is visible through-

outthe package, and ifthe nameisexp orted (it startswit h an upp er-case letter), it’saccessible

from other packages as wel l.

To i llustratetyp e de clarat ions,let’stur n thedif ferenttemperaturescales into dif ferenttyp es:

gopl.io/ch2/tempconv0

// Package tempconv performs Celsius and Fahrenheit temperature computations.

package tempconv

import "fmt"

type Celsius float64

type Fahrenheit float64

const (

AbsoluteZeroC Celsius = -273.15

FreezingC Celsius =0

BoilingC Celsius =100

)

func CToF(c Celsius) Fahrenheit { return Fahrenheit(c*9/5 + 32) }

func FToC(f Fahrenheit) Celsius { return Celsius((f - 32) * 5 / 9) }

Thispackagedeﬁnestwo typ es, Celsius and Fahrenheit,for the two units oftemperature.

Ev enthoug h both havethe sameunderly ing typ e, float64,the y arenot the sametyp e,sothe y

cannot becompare d or com bine d in arit hmeticexpressions.Distinguishingthe typ es makesit

possible toavoid erro rslikeinadver tentlycom biningtemperatures in the two dif ferentscales;

an explicittyp e conv ersion li ke Celsius(t) or Fahrenheit(t) is required toconvert from a

float64. Celsius(t) and Fahrenheit(t) areconversions,not functioncal ls. They don’t

ch ange the value orrepresent ation inany way,but the y make the change ofmeaningexplicit.

On the other hand, the functions CToF and FToC conv ert bet weenthe two scales; the y do

return dif ferentvalues.

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

40 CHAPTER 2. PROGRAM STRUCTURE

Fo r ever y type T,there isacor respondingconversionoperat ion T(x) that conv ertsthe value x

to typ e T.Aconv ersionfro m on e type toanother isallow ed if bot h have the sameunderly ing

type,orifbot h areunnamed point ertyp es that point tovar iables ofthe sameunderly ing typ e;

thes e conv ersions change the typ e butnot the represent ation of the value.If x is assig nable to

T,aconversionisper mitt edbut isusu allyredundant,

Conv ersions are als o al lowe d betweennumer ic types, andbet weenstr ing and som e slice typ es,

as wewill see inthe next chapt er. These conversions may change the represent ation of the

value.For ins tance,convertingaﬂoating-p ointnumbertoanint egerdis cards any frac tional

part,and convertingastr ing toa[]byte slice allo cates a copy ofthe str ing dat a. In any cas e,a

conv ersionnever fails atrun time.

Theunderly ing typ e of a named typ e deter mines its str uctureand represent ation,and als o the

setofint rinsic operat ions itsup por ts, whichare the sameasifthe underly ing typ e hadbeen

us eddirec tly.Thatmeans thatarithmeticoperator s work the samefor Celsius and Fahren-

heit as the y do for float64, as you mig htexp ect.

fmt.Printf("%g\n", BoilingC-FreezingC) // "100" °C

boilingF := CToF(BoilingC)

fmt.Printf("%g\n", boilingF-CToF(FreezingC)) // "180" °F

fmt.Printf("%g\n", boilingF-FreezingC) // compile error: type mismatch

Comp arisonoperator s li ke == and < canals o be usedtocompare a value ofanamed typ e to

anot her ofthe sametyp e,ortoavalue ofthe underly ing typ e.But two values ofdif ferent

name d typescannot becompare d direc tly :

var c Celsius

var f Fahrenheit

fmt.Println(c == 0) // "true"

fmt.Println(f >= 0) // "true"

fmt.Println(c == f) // compile error: type mismatch

fmt.Println(c == Celsius(f)) // "true"!

No tethe lastcas e carefully. Inspit e of itsname, the typ e conv ersion Celsius(f) do es not

ch ange the value ofits argument, justits typ e.The testistruebecause c and f arebot h zero.

Anamed typ e mayprovide not ation alconvenience if ithelps avoidwriting out complex typ es

ov erand overagain. Theadvantage issmall whenthe underly ing typ e is simplelike float64,

butbig for complic ated typ es, as wewill see whenwedis cussstr ucts.

Name d typesals o make itpossibletodeﬁne newbeh avior s forvalues ofthe typ e.These

behavior s areexpress edasaset offunctions ass oci ated wit h thetyp e,cal le d thetyp e’s metho ds.

We’ll look at met hods in det ail in Chapt er6but will give a taste ofthe mech anism here.

Thedeclarat ionbelow,inwhichthe Celsius parameter c appearsbeforethe functionname,

asso ciateswit h the Celsius type a met hod named String that retur ns c’snumer ic value

fo llowe d by °C:

func (c Celsius) String() string { return fmt.Sprintf("%g°C", c) }

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

SECTION 2.6. PACKAGES AND FILES 41

Many typ es de clare a String method ofthisfor m becaus e it control s howvalues ofthe typ e

appear whenprint edasastr ing bythe fmt package, aswewill see inSec tion 7.1.

c:=FToC(212.0)

fmt.Println(c.String()) // "100°C"

fmt.Printf("%v\n", c) // "100°C"; no need to call String explicitly

fmt.Printf("%s\n", c) // "100°C"

fmt.Println(c) // "100°C"

fmt.Printf("%g\n", c) // "100"; does not call String

fmt.Println(float64(c)) // "100"; does not call String

2.6. Packages and Files

Packages in Goser vethe samepur pos es as librar ies or modules in other langu ages, sup por ting

mo dular ity,enc apsulat ion, sep aratecompi lat ion, andreuse.The sourcecodefor a package

resides in one ormore .go ﬁles, usu allyinadirec tor y whos e name end s with theimp ort pat h;

forins tance,the ﬁles ofthe gopl.io/ch1/helloworld packageare store d in direc tor y

$GOPATH/src/gopl.io/ch1/helloworld.

Each packageser ves as a sep arate name space forits decl arat ions.Wit hin the image package,

forexample, the identiﬁer Decode refers to a dif ferentfunctionthandoesthe sameidentiﬁer

in the unicode/utf16 package. Torefer toafunctionfro m outside its package, wemust

qu ali fy theidentiﬁer tomakeexplicitwhether wemean image.Decode or utf16.Decode.

Packages also let ushide infor mat ionbycontrol lingwhichnames arevisible outside the pack-

age, or expor ted.InGo, a simplerulegov ernswhichidentiﬁersare exp orted and whichare

not: expor ted identiﬁersstart wit h an upp er-case letter.

To illustratethe basics, sup pos e that our temperatureconversionsof twarehas becom e popu lar

andwewanttomakeitavai lable tothe Gocommunity asanew package. How dowedothat?

Let’scre ate a packagecal le d gopl.io/ch2/tempconv,avar iat iononthe pre vious example.

(Herewe’ve made an exception toour usu alruleofnumbering examples in sequence,sothat

thepackagepat h canbemorerealist ic.) Thepackageits elf isstore d in two ﬁles toshowhow

de clarat ions insep arate ﬁles ofapackageare accessed; in reallife, a tinypackagelikethis

wouldneed onlyone ﬁle.

We haveput the declarat ions ofthe typ es, their cons tants, andtheir met hodsin tempconv.go:

gopl.io/ch2/tempconv

// Package tempconv performs Celsius and Fahrenheit conversions.

package tempconv

import "fmt"

type Celsius float64

type Fahrenheit float64

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

42 CHAPTER 2. PROGRAM STRUCTURE

const (

AbsoluteZeroC Celsius = -273.15

FreezingC Celsius =0

BoilingC Celsius =100

)

func (c Celsius) String() string {return fmt.Sprintf("%g°C", c) }

func (f Fahrenheit) String() string { return fmt.Sprintf("%g°F", f) }

andthe conversionfunctions in conv.go:

package tempconv

// CToF converts a Celsius temperature to Fahrenheit.

func CToF(c Celsius) Fahrenheit { return Fahrenheit(c*9/5 + 32) }

// FToC converts a Fahrenheit temperature to Celsius.

func FToC(f Fahrenheit) Celsius { return Celsius((f - 32) * 5 / 9) }

Each ﬁle startswit h a package de clarat ionthatdeﬁnesthe packagename. Whenthe package

is imp orted,its members are referred toas tempconv.CToF andsoon. Package-le vel names

li kethe typ es andcon stantsdeclare d in one ﬁle ofa packageare visible toall the other ﬁles of

thepackage, asifthe sourcecodewereall inasingle ﬁle.Not e that tempconv.go imports fmt,

but conv.go do es not, because itdoesnot use anythingfro m fmt.

Becaus e thepackage-le vel const namesbeg in with upp er-case letters,the y to o areaccessible

with qualiﬁed names like tempconv.AbsoluteZeroC:

fmt.Printf("Brrrr! %v\n", tempconv.AbsoluteZeroC) // "Brrrr! -273.15°C"

To c onvert a Cel siu s temp eraturetoFahren heitinapackagethatimp orts gopl.io/ch2/temp-

conv,wecan write the fol low ing code:

fmt.Println(tempconv.CToF(tempconv.BoilingC)) // "212°F"

The doc comment (§10.7.4) immediate lypre cedingthe package de clarat iondocuments the

packageasawhole.Convent ion ally, itshouldstart wit h asummar y sent enceinthe sty le

illustrated.Onlyone ﬁle in eachpackageshouldhaveapackagedoc comment.Extensive doc

comments areoften place d in a ﬁle oftheir own,convent ion allycal le d doc.go.

Exercis e 2.1: Addtyp es, cons tants, andfunctions to tempconv forpro cessingtemperatures in

theKelvinscale,where zeroKelvinis −273.15°C andadif ference of1Khas the samemag ni-

tude as 1°C.

2.6.1. Imports

Wi thin a Goprogram, every packageisidentiﬁed byaunique str ing cal le d its import pat h.

Thes e arethe str ingsthatapp ear in an import de clarat ionlike "gopl.io/ch2/tempconv".

Thelangu agespeciﬁcation doesn’tdeﬁne where these str ingscom e from or whatthe y me an;

it’suptothe tools toint erpretthem. Whenusingthe go to ol(Chapter10), an import pat h

denot esadirec tor y cont ainingone ormoreGosourceﬁles thattoget her makeupthe package.

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

SECTION 2.6. PACKAGES AND FILES 43

In addition toits import pat h, each packagehas a pack age nam e,whichisthe short (andnot

ne cessarily unique) namethatapp earsinits package de clarat ion. Byconvent ion,apackage’s

name match esthe lastseg mentofits import pat h, making iteasy topre dic t that the package

name of gopl.io/ch2/tempconv is tempconv.

To use gopl.io/ch2/tempconv,wemustimp ort it:

gopl.io/ch2/cf

// Cf converts its numeric argument to Celsius and Fahrenheit.

package main

import (

"fmt"

"os"

"strconv"

"gopl.io/ch2/tempconv"

)

func main() {

for _, arg := range os.Args[1:] {

t, err := strconv.ParseFloat(arg, 64)

if err != nil {

fmt.Fprintf(os.Stderr, "cf: %v\n", err)

os.Exit(1)

}

f:=tempconv.Fahrenheit(t)

c:=tempconv.Celsius(t)

fmt.Printf("%s = %s, %s = %s\n",

f, tempconv.FToC(f), c, tempconv.CToF(c))

}

Theimp ort declarat ionbinds a short nametothe imp orted packagethatmay beusedtorefer

to its cont entsthroughoutthe ﬁle.The import ab ove lets usrefer tonames wit hin

gopl.io/ch2/tempconv by usingaqu aliﬁed ident iﬁer li ke tempconv.CToF.Bydefau lt, the

short nameisthe packagename—tempconv in thiscas e—but an imp ort declarat ionmay

sp ecif y an alternat ive nametoavoid a conﬂic t (§10.3).

The cf prog ram conv ertsasinglenumer ic command-lineargumenttoits value in bot h

Celsiu s andFahren heit:

$gobuild gopl.io/ch2/cf

$./cf 32

32°F = 0°C, 32°C = 89.6°F

$./cf 212

212°F = 100°C, 212°C = 413.6°F

$./cf -40

-40°F = -40°C, -40°C = -40°F

It isanerror toimp ort a packageand thennot refer toit. Thische ckhelps eliminatedep en-

dencies thatbecom e unnecessary asthe codeevo l ves, although itcan beanuis ancedur ing

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

44 CHAPTER 2. PROGRAM STRUCTURE

debugging, since commenting out a lineofcodelike log.Print("got here!") mayremov e

thesolereference tothe packagename log,causingthe compi ler toemitanerror.Inthissit-

uation,you need tocomment out or deletethe unnecessary import.

Bett erstill,use the golang.org/x/tools/cmd/goimports to ol, whichaut omat ical lyins erts

andremov espackages fro m theimp ort declarat ionasnecessary ; most editors can beconﬁg-

ured torun goimports each timeyou saveaﬁle.Likethe gofmt to ol, itals o pretty-printsGo

source ﬁles in the canonic al format.

Exercis e 2.2: Wr ite a general-pur pos e unit-conv ersionprogram analogou s to cf that reads

numb ers fro m itscommand-linearguments orfro m thestand ard inp utifthere are noargu-

ments, andconvertseachnumberint o units like temperatureinCel siu s andFahren heit,

lengt h in feetand meters,weig ht in pound s andkilog rams, andthe like.

2.6.2. PackageInitialization

Packageinitializat ionbeg insbyinitializingpackage-le vel var iables in the order in whichthe y

aredeclare d,exceptthatdep endencies areres olved ﬁrs t:

var a = b + c // a initialized third, to 3

var b = f() // b initialized second, to 2, by calling f

var c = 1 // c initialized first, to 1

func f() int { return c + 1 }

If the packagehas multiple .go ﬁles, the y areinitialize d in the order in whichthe ﬁles are

giventothe compi ler ; the go to olsor ts .go ﬁles bynamebeforeinv oking the compi ler.

Each var iable decl are d at packagelevel startslifewit h thevalue ofits initializer expression, if

any, but for som e var iables, like tables ofdat a, an initializer expressionmay not bethe simplest

way toset its initial value.Inthatcas e,the init func tionmechanism may besimpler.Any

ﬁle may cont ain anynumberoffunctions whose declarat ionisjust

func init() { /* ... */ }

Such init func tions can’tbecal le d or reference d,but other wis e they are nor mal functions.

Wi thin each ﬁle, init func tions are aut omat ical lyexe cut edwhenthe program starts, in the

order in whichthe y aredeclare d.

Onepackageisinitialize d at a time, inthe order ofimp ortsinthe program, dependencies ﬁrs t,

so a package p importing q canbesurethat q is fullyinitialize d before p’sinitializat ionbeg ins.

Init ializat ionpro ceedsfro m thebot tom up;the main packageisthe lasttobeinitialize d.In

this manner,all packages arefullyinitialize d before the app lic ation’s main func tionbeg ins.

Thepackagebelow deﬁnesafunction PopCount that retur nsthe numberofset bits, thatis,

bitswhose value is1,inauint64 value,whichiscal le d its popu lat ion count.Itusesan init

func tiontopre compute a table ofresults, pc,for eachpossible8-bit value sothatthe PopCount

func tionneedn’ttake64steps but can justretur n thesum ofeig httable lookups. (Thisisdeﬁ-

nite ly not thefastest algor it hmfor count ing bits, but it’sconvenientfor illustrat ing init

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

SECTION 2.7. SCOPE 45

func tions,and for showing how topre compute a table ofvalues, whichisoften a useful

prog rammingtechnique.)

gopl.io/ch2/popcount

package popcount

// pc[i] is the population count of i.

var pc [256]byte

func init() {

for i := range pc {

pc[i] = pc[i/2] + byte(i&1)

}

// PopCount returns the population count (number of set bits) of x.

func PopCount(x uint64) int {

return int(pc[byte(x>>(0*8))] +

pc[byte(x>>(1*8))] +

pc[byte(x>>(2*8))] +

pc[byte(x>>(3*8))] +

pc[byte(x>>(4*8))] +

pc[byte(x>>(5*8))] +

pc[byte(x>>(6*8))] +

pc[byte(x>>(7*8))])

}

No tethatthe range lo opin init us esonlythe index; the value isunnecessary and thu s ne e d

notbeinclude d.The loopcou ldals o have beenwritt enas

for i, _ := range pc {

We’ll see other usesof init func tions inthe next sec tion and inSec tion 10.5.

Exercis e 2.3: Re writ e PopCount to use a loopins teadofasingleexpression. Compare the per-

formance ofthe two versions.(Section11.4 shows how tocompare the per for mance ofdif fer-

entimp lementation s systematically.)

Exercis e 2.4: Wr ite a versionof PopCount that counts bitsbyshif tingits argumentthrough 64

bit position s,testing the rig htmostbit eachtime. Compare its per for mance tothe table-

lo oku p version.

Exercis e 2.5: Theexpression x&(x-1) clears the rig htmostnon-zerobit of x.Write a version

of PopCount that counts bitsbyusingthisfac t, andass ess its per for mance.

2.7. Scope

Adeclarat ionass oci ates a namewit h aprogram entity,suchasafunctionoravar iable.The

scop e of a declarat ionisthe par t of the sourcecodewhere a use ofthe declare d name refersto

that decl arat ion.

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

46 CHAPTER 2. PROGRAM STRUCTURE

Don’tconfuse scop e with lifet ime. The scope ofadeclarat ionisareg ionofthe program text;

it isacompi le-t imepro per ty. The lifet imeofavar iable isthe range oftimedur ingexe cut ion

when the var iable can berefer red tobyother par ts of the program; itisarun-t imepro per ty.

Asyntactic bl o ck is a sequence ofstatementsenclos edinbraces like those thatsur round the

bodyofafunctionorloop. A name declare d inside a synt actic blo ckisnot visible outside that

block. Theblo ckenclos es itsdeclarat ions and deter mines their scope.Wecan generalize this

notion of blo cks toinclude other gro upingsofdeclarat ions thatare not explicitlysur rounded

by braces in the sourcecode; we’ll cal l them all lexicalblocks.There isalexic al blockfor the

entire sourcecode, cal le d the univ ers e bl o ck;for eachpackage; for each ﬁle; for each for, if,

and switch st atement;for eachcas e in a switch or select st atement;and,ofcours e, foreach

explicitsyntactic blo ck.

Adeclarat ion’slexic al blockdeter mines its scope,whichmay belarge orsmall.The declara-

tion s of bui lt-in typ es, func tions,and con stantslike int, len,and true areinthe univers e

blockand can berefer red tothroughoutthe ent ire program. Declarat ions outside anyfunc-

tion,thatis, at pack age level,can berefer red tofro m any ﬁle in the samepackage. Imp orted

packages, suchas fmt in the tempconv example, are declare d at the ﬁle level,sothe y canbe

referred tofro m thesame ﬁle,but not fro m anot her ﬁle in the samepackagewit houtanother

import.Manydeclarat ions,likethatofthe var iable c in the tempconv.CToF func tion, are

local,sothe y canberefer red toonlyfro m within the samefunctionorperhaps justapar t of it.

Thescope ofacontrol-ﬂow lab el, asusedby break, continue,and goto st atements, isthe

entire enclosingfunction.

Aprogram may cont ain multipledeclarat ions ofthe samenamesolon g as eachdeclarat ionis

in a different lexic al block. For example, you can decl are a local variablewit h thesamename

as a package-le vel var iable.Or, asshown inSec tion 2.3.3, you can decl are a functionparame-

tercal le d new,eventhoug h afunctionofthisnameispre declare d in the univers e block. Don’t

ov erdoit, thoug h; thelargerthe scope ofthe redeclarat ion, the more likelyyou are tosur prise

thereader.

Wh enthe compi ler encounters a reference toaname, itlooks for a declarat ion, startingwit h

theinner mostenclosinglexic al blockand wor kinguptothe univers e block. Ifthe compi ler

ﬁnd s no declarat ion, itrep ortsan ‘‘unde clare d name’’ er ror.Ifanameisdeclare d in bot h an

outerblo ckand aninner blo ck, the inner decl arat ionwill befound ﬁrs t.Inthatcas e,the

inner decl arat ionissaid to sh a dow or hi d e theout erone,mak ingitinaccessible:

func f() {}

var g = "g"

func main() {

f:="f"

fmt.Println(f) // "f"; local var f

shadows

package-level func f

fmt.Println(g) // "g"; package-level var

fmt.Println(h) // compile error: undefined: h

}

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

SECTION 2.7. SCOPE 47

Wi thin a function, lexical blo cks may beneste d to arbit rar y depth,soone local decl arat ioncan

sh adowanother.Mostblo cks arecre ate d by control-ﬂow con str uctslike if st atementsand

for lo ops. Theprogram below has three dif ferentvar iables cal le d x becaus e each declarat ion

appears in a dif ferentlexic al block. (Thisexampleillustrates scope rules, not good sty le!)

func main() {

x:="hello!"

for i := 0; i < len(x); i++ {

x:=x[i]

if x != '!' {

x:=x+'A' - 'a'

fmt.Printf("%c", x) // "HELLO" (one letter per iteration)

}

Theexpressions x[i] and x+'A' - 'a' each refer toadeclarat ionof x from anout erblo ck;

we’ll explain thatinamom ent.(No tethatthe latterexpressionis not equivalentto uni-

code.ToUpper.)

As mention edabove , notall lexic al blocks correspond toexplicitbrace-delimite d sequences of

st atements; som e aremerelyimp lie d.The for lo opabove cre atestwo lexic al blocks: the

explicitblo ckfor the loopbody, and animp licitblo ckthataddition allyenclos es thevar iables

de clare d by the initializat ionclaus e,suchas i.The scope ofavar iable decl are d in the imp licit

blockisthe con dit ion,post-statement (i++), andbodyofthe for st atement.

Theexamplebelow als o hasthree var iables named x,eachdeclare d in a different blo ck—on e

in the functionbody, one inthe for st atement’sblo ck, andone inthe loopbody—butonlytwo

of the blo cks areexplicit:

func main() {

x:="hello"

for _, x := range x {

x:=x+'A' - 'a'

fmt.Printf("%c", x) // "HELLO" (one letter per iteration)

}

Like for lo ops, if st atementsand switch st atementsals o create imp licitblo cks in addition to

their bodyblo cks. Thecode in the fol low ing if-else ch ain shows the scope of x and y:

if x := f(); x == 0 {

fmt.Println(x)

}else if y := g(x); x == y {

fmt.Println(x, y)

}else {

fmt.Println(x, y)

}

fmt.Println(x, y) // compile error: x and y are not visible here

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

48 CHAPTER 2. PROGRAM STRUCTURE

Thesecon d if st atement isneste d within the ﬁrs t,sovar iables decl are d within the ﬁrs t st ate-

ment’sinitializer arevisible wit hin the secon d.Simi lar rules apply toeachcas e of a switch

st atement:there isablo ckfor the con dit ion and a blo ckfor eachcas e body.

At the packagelevel,the order in whichdeclarat ions app ear has noeffec t on their scope,soa

de clarat ionmay refer toits elf ortoanother thatfol lowsit, letting usdeclare recursiveor

mu tuallyrec ursivetyp es andfunctions.The compi ler will rep ort anerror ifacon stant orvar i-

able decl arat ionreferstoits elf,how ever.

In thisprogram:

if f, err := os.Open(fname); err != nil { // compile error: unused: f

return err

}

f.ReadByte() // compile error: undefined f

f.Close() // compile error: undefined f

thescope of f is justthe if st atement,so f is not accessibletothe statementsthatfol low,

resu lting incompi ler erro rs. Dep endingonthe compi ler,you may get an addition alerror

reportingthatthe local variable f was never used.

Thus itisoften necessary todeclare f before the con dit ion sothatitisaccessibleafter :

f, err := os.Open(fname)

if err != nil {

return err

}

f.ReadByte()

f.Close()

Yo u maybetempt edtoavoid decl aring f and err in the out erblo ckbymov ing the cal lsto

ReadByte and Close inside an else block:

if f, err := os.Open(fname); err != nil {

return err

}else {

// f and err are visible here too

f.ReadByte()

f.Close()

}

butnor mal prac tice in Goistodealwit h theerror inthe if blockand thenretur n, so thatthe

successf ulexe cut ion pat h is not indente d.

Sh ort var iable decl arat ions demandanawarenessofscope.Con sider the program below,

whichstartsbyobt ainingits cur rentwor kingdirec tor y andsav ingit in a package-le vel var i-

able.Thiscou ldbedon e by cal ling os.Getwd in function main,but it mig htbebettertosep a-

ratethiscon cer n from the primary log ic, especi ally iffai lingtoget the direc tor y is a fat al er ror.

Thefunction log.Fatalf pr intsamessage and cal ls os.Exit(1).

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

SECTION 2.7. SCOPE 49

var cwd string

func init() {

cwd, err := os.Getwd() // compile error: unused: cwd

if err != nil {

log.Fatalf("os.Getwd failed: %v", err)

}

Sinceneither cwd nor err is already declare d in the init func tion’sblo ck, the := st atement

de claresbot h of themaslocal variables. Theinner decl arat ionof cwd makesthe out erone

inaccessible, sothe statement doesnot updatethe package-le vel cwd var iable as intended.

CurrentGocompi lersdetec t that the local cwd var iable isnever usedand rep ort thisasan

er ror,but the y arenot str ictly required toper for m this che ck. Fur thermore, a minor change ,

such asthe addition of a log gingstatement thatreferstothe local cwd woulddefeatthe che ck.

var cwd string

func init() {

cwd, err := os.Getwd() // NOTE: wrong!

if err != nil {

log.Fatalf("os.Getwd failed: %v", err)

}

log.Printf("Working directory = %s", cwd)

}

Theglobal cwd var iable remainsuninitialize d,and the app arent lynor mal log out put

obfuscates the bug .

Thereare a numberofways todealwit h this pot ent ial pro blem. Themostdirec t is toavoid :=

by declaring err in a sep arate var de clarat ion:

var cwd string

func init() {

var err error

cwd, err = os.Getwd()

if err != nil {

log.Fatalf("os.Getwd failed: %v", err)

}

We’ve now seenhow packages, ﬁles, decl arat ions,and statementsexpress the str uctureof

prog rams. Inthe next two chapt ers,we’ll look at the str uctureofdat a.

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

This page intentionally left blank

From the Library of YIGUANG HU

ptg16105617

Basic Data Types

It’sall bits at the bot tom,ofcours e, butcomputers operatefundament allyonﬁxe d-size num-

bers cal le d word s,whichare int erprete d as integers,ﬂoating-p ointnumbers,bit sets, ormem-

or y addresses, thencom bine d into largeraggregates thatrepresent packets, pixel s,por tfo lios,

poetr y,and everythingels e.Gooffersavar ietyofways toorganize dat a, with a spectrumof

data typ es that at one end match the featuresofthe hardwareand atthe other endprovide

what programmersneed toconvenientlyrepresent complic ated dat a st ruc tures.

Go’styp es fal l into fourcategor ies: basi c ty pes, ag gre gat e ty pes, reference types,and interface

ty pes.Basic typ es, thetopic ofthischapt er, include numbers,str ings, andboole ans. Aggregate

types—ar rays (§4.1) andstr ucts(§4.4)—form morecomplic ated dat a typesbycom biningval-

ues ofseveral simpler ones. Reference typ es areadiverse gro upthatincludes point ers (§2.3.2),

slices (§4.2), maps (§4.3), functions (Chapter5), andchannel s (C hapter8), but whatthe y have

in common isthatthe y al l refertoprogram variables orstate in direc tly,sothatthe effec t of an

op erat ionapp lie d to one reference isobs erve d by all copies ofthatreference.Final ly, we’ll tal k

ab out int erface typ es in Chapt er7.

3.1. Int egers

Go’snumer ic data typ es include several sizes ofint egers,ﬂoating-p ointnumbers,and complex

numb ers.Eachnumer ic type deter mines the size andsig nedness ofits values. Let’sbeg in with

integers.

Go provides bot h sig ned and unsig ned int egerarithmetic. Thereare fourdistinc t sizes of

sig ned int egers—8, 16, 32, and64bits—repres ente d by the typ es int8, int16, int32,and

int64,and cor respondingunsig ned versions uint8, uint16, uint32,and uint64.

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

52 CHAPTER 3. BASIC DATATYPES

Thereare als o twotyp es called just int and uint that arethe natural ormostefﬁcientsize for

sig ned and unsig ned int egers onapar tic ularplatfor m; int is byfar the most widely used

numerictyp e.Bot h thes e typeshavethe samesize,eit her 32 or64bits, but onemustnot make

assump tions about which; different compi lersmay makedif ferentchoices evenonidentical

hard ware.

Thetyp e rune is ansynonym for int32 andconvent ion allyindic ates thatavalue isaUnico de

co de point. Thetwo names may beusedint erc hange ably.Simi larly,the typ e byte is ansyn-

onym for uint8,and emp hasizes thatthe value isapie ce of raw dat a rat her thanasmall

numericquant ity.

Final ly, there isanunsig ned int egertyp e uintptr,whose width isnot speciﬁed but issufﬁ-

cienttoholdall the bitsofapoint ervalue.The uintptr type isusedonlyfor low-le vel

prog ramming, suchasatthe bound ary ofaGoprogram wit h aClibrar y or anoperat ingsys-

tem. We’ll see examples ofthiswhenwedealwit h the unsafe package in Chapt er13.

Regardlessoftheir size, int, uint,and uintptr aredif ferenttyp es from their explicitlysize d

siblings. Thus int is not the sametyp e as int32,evenifthe natural size ofint egers is32bits,

andanexplicitconversionisrequired touse an int value where an int32 is needed,and vice

vers a.

Signed numbers are repres ente d in 2’s-comp lementfor m,inwhichthe hig h-order bit is

reserved for the sig n of the numberand the range ofvalues ofan n-bit numberisfro m −2

n−1

to 2

n−1

−1. Unsig ned int egers use the full range ofbitsfor non-negat ive values andthu s have

therange 0 to2

−1. For ins tance,the range of int8 is −128 to127, where asthe range of

uint8 is 0 to255.

Go’sbinar y op erator s forarithmetic, logic, andcomparisonare liste d here inorder ofdecre as-

ingpre cedence:

*/%<<>>&&^

+-|^

== != < <= > >=

Thereare onlyﬁve level s of precedence for binary operator s.Operator s at the samelevel as-

so ciate to the lef t, so parenthes es mayberequired for clarity,ortomakethe operator s evaluate

in the int ended order in an expressionlike mask & (1 << 28).

Each operator inthe ﬁrs t twolines ofthe table abov e,for ins tance +,has a corresponding

assig nment operator li ke += that may beusedtoabbre viate anassig nmentstatement.

Theint egerarithmeticoperator s +, -, *,and / maybeapp lie d to int eger, ﬂoating-p oint, and

comp lex numbers,but the remainder operator % applies onlytoint egers.The beh avior of % for

negat ive numbers var ies acrossprogramminglangu ages. InGo, the sig n of the remainder is

always the sameasthe sig n of the div idend, so -5%3 and -5%-3 arebot h -2.The beh avior of /

dep ends onwhether its operands are int egers,so 5.0/4.0 is 1.25,but 5/4 is 1 becaus e integer

divisiontrunc atesthe resulttowardzero.

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

SECTION 3.1. INTEGERS 53

If the resultof an arithmeticoperat ion, whether signe d or unsig ned,has morebitsthancan be

repres ente d in the resulttyp e,itissaid to ov erﬂow.The hig h-order bitsthatdonot ﬁtare

si lentlydis carde d.Ifthe originalnumberisasig ned typ e,the resultcou ldbenegat ive ifthe

lef tmost bit is a 1, as in the int8 examplehere:

var u uint8 = 255

fmt.Println(u, u+1, u*u) // "255 0 1"

var i int8 = 127

fmt.Println(i, i+1, i*i) // "127 -128 1"

Tw o integers ofthe sametyp e maybecompare d usingthe binar y comp arisonoperator s

below; the typ e of a comparisonexpressionisaboole an.

== equalto

!= notequ alto

< lessthan

<= lessthanorequ alto

> greaterthan

>= greaterthanorequ alto

In fac t, al l values ofbasic typ e—boole ans, numbers,and str ings—are comp arab le,meaning

that two values ofthe sametyp e maybecompare d usingthe == and != op erator s.Fur ther-

more , integers,ﬂoating-p ointnumbers,and str ingsare ordere d by the comparisonoperator s.

Thevalues ofmanyother typ es arenot comparable,and no other typ es areordered.Aswe

encountereachtyp e,we’ll present the rules gov erningthe comp arab ility of itsvalues.

Thereare als o unar y addition and subtrac tionoperator s:

+ unar y positive (no effec t)

- unar y negat ion

Fo r integers, +x is a shorthand for 0+x and -x is a shorthand for 0-x;for ﬂoating-p ointand

comp lex numbers, +x is just x and -x is the negat ionof x.

Go als o prov ides the fol low ing bit w ise binar y op erator s,the ﬁrs t four ofwhichtre attheir op-

erands asbit patternswit h no con ceptofarithmeticcar ryorsig n:

& bit w ise AND

| bit w ise OR

^ bit w ise XOR

&^ bit cle ar (AND NOT)

<< lef t shif t

>> rightshif t

Theoperator ^ is bit w ise exc lusiveOR(XO R)whenusedasabinar y op erator,but whenused

as a unar y preﬁxoperator it isbit w ise negat ionorcomplement; thatis, itretur nsavalue wit h

each bit inits operandinv erted.The &^ op erator isbit cle ar (AND NOT): in the expression

z=x&^y,eachbit of z is 0 ifthe cor respondingbit of y is 1;other wis e it equ als the cor-

resp ondingbit of x.

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

54 CHAPTER 3. BASIC DATATYPES

Thecodebelow shows how bit w ise operat ions can beusedtoint erpretauint8 value as a

comp act and efﬁcientset of8indep endentbits. Ituses Printf’s %b verb toprint a numb er’s

binary dig its; 08 mo diﬁes %b (an adverb!) topad the resultwit h zeros toexac tly 8 dig its.

var x uint8 = 1<<1 | 1<<5

var y uint8 = 1<<1 | 1<<2

fmt.Printf("%08b\n", x) // "00100010", the set {1, 5}

fmt.Printf("%08b\n", y) // "00000110", the set {1, 2}

fmt.Printf("%08b\n", x&y) // "00000010", the intersection {1}

fmt.Printf("%08b\n", x|y) // "00100110", the union {1, 2, 5}

fmt.Printf("%08b\n", x^y) // "00100100", the symmetric difference {2, 5}

fmt.Printf("%08b\n", x&^y) // "00100000", the difference {5}

for i := uint(0); i < 8; i++ {

if x&(1<<i) != 0 { // membership test

fmt.Println(i) // "1", "5"

}

fmt.Printf("%08b\n", x<<1) // "01000100", the set {2, 6}

fmt.Printf("%08b\n", x>>1) // "00010001", the set {0, 4}

(S ection6.5 shows an implementation of int egersets thatcan bemuchbig ger thanabyte.)

In the shif t op erat ions x<<n and x>>n,the n op eranddeter mines the numberofbit position s

to shif t andmustbeunsig ned;the x op erandmay beunsig ned orsig ned.Arithmetically, a lef t

shif t x<<n is equivalenttomultiplic ationby2

andarig htshif t x>>n is equivalenttothe ﬂoor

of div isionby2

Left shif ts ﬁll the vac ated bitswit h zeros, as do rig htshif ts of unsig ned numbers,but rig ht

shif ts of sig ned numbers ﬁll the vac ated bitswit h copies ofthe sig n bit.For thisreason, itis

importanttouse unsig ned arithmeticwhenyou’re tre ating anint eger as a bit pattern.

Although Goprovides unsig ned numbers and arithmetic, wetendtouse the sig ned int form

even for quant ities thatcan’tbenegat ive , such asthe lengt h of anarray,thoug h uint mig ht

seem a moreobv iou s ch oice.Indeed,the bui lt-in len func tionretur nsasig ned int,asinthis

lo opwhichannounces prize medals in reverse order :

medals := []string{"gold", "silver", "bronze"}

for i := len(medals) - 1; i >= 0; i-- {

fmt.Println(medals[i]) // "bronze", "silver", "gold"

}

Thealt ernat ive wouldbecal amitous.If len returned anunsig ned number, then i to o would

be a uint,and the con dit ion i>=0wouldalways betruebydeﬁnition.After the thirditera-

tion,inwhich i==0,the i-- st atement wou ldcause i to becom e not −1, but the maximum

uint value (for example, 2

−1), andthe evaluation of medals[i] wouldfai l at run time, or

pani c (§5.9), byatt emp tingtoaccess an elementoutside the bound s of the slice.

Fo r this reason, unsig ned numbers tend tobeusedonlywhentheir bit w ise operator s or

peculi ar ar ithmeticoperator s arerequired,aswhenimp lementing bit sets, parsingbinar y ﬁle

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

SECTION 3.1. INTEGERS 55

formats, orfor hashingand crypt ography.The y aretypic ally not usedfor merelynon-negat ive

qu ant ities.

In general,anexplicitconversionisrequired toconvert a value fro m on e type toanother,and

binary operator s forarithmeticand log ic (exceptshif ts) mu sthaveoperands ofthe sametyp e.

Although thisocc asionallyresults in longerexpressions,itals o eliminates a whole class of

problemsand makes programseasier tounderst and.

As anexamplefami liar fro m ot her cont exts, consider thissequence:

var apples int32 = 1

var oranges int16 = 2

var compote int = apples + oranges // compile error

At tempt ing tocompi lethese three declarat ions pro duces an erro r mess age:

invalid operation: apples + oranges (mismatched types int32 and int16)

Thistyp e mismatch can be ﬁxe d in several ways, mostdirec tly byconvertingeverythingtoa

common typ e:

var compote = int(apples) + int(oranges)

As des crib edinSec tion 2.5, for every typ e T,the conversionoperat ion T(x) conv ertsthe value

x to typ e T if the conversionisallow ed. Manyint eger-to-integerconversions donot ent ail any

ch ange invalue; the y ju sttel l thecompi ler how toint erpretavalue.But a conversionthatnar-

rows a big integerint o asmaller one,oraconversionfro m integertoﬂoating-p ointorvice

vers a,may change the value orlos e precision:

f:=3.141 // a float64

i:=int(f)

fmt.Println(f, i) // "3.141 3"

f=1.99

fmt.Println(int(f)) // "1"

Flo at to int egerconversiondis cards any frac tionalpar t, tr unc ating towardzero. You should

avoidconversions inwhichthe operandisout of range for the targettyp e,because the beh av-

iordep ends onthe imp lementation:

f:=1e100 // afloat64

i:=int(f) // result is implementation-dependent

Integerlit eralsofany size andtyp e canbewritt enasordinar y de cimal numbers,orasoctal

numb ers ifthe y beginwit h 0,asin 0666,orashexade cimal if the y beginwit h 0x or 0X,asin

0xdeadbeef.Hex digitsmay beupp erorlow ercas e.Nowad ays octalnumbers seemtobe

us edfor exac tly one pur pos e—ﬁle per missions onPOSIX systems—buthexade cimal numbers

arewidely usedtoemp hasize the bit pattern ofanumberoverits numer ic value.

Wh enprint ing numbers usingthe fmt package, wecan cont rol the radix andfor mat wit h the

%d, %o,and %x verbs, as shown inthisexample:

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

56 CHAPTER 3. BASIC DATATYPES

o:=0666

fmt.Printf("%d %[1]o %#[1]o\n", o) // "438 666 0666"

x:=int64(0xdeadbeef)

fmt.Printf("%d %[1]x %#[1]x %#[1]X\n", x)

// Output:

// 3735928559 deadbeef 0xdeadbeef 0XDEADBEEF

No tethe use oftwo fmt tr icks. Usu allyaPrintf format str ing containingmultiple % verbs

wouldrequirethe samenumberofext ra op erands,but the [1] ‘‘adverbs’’ af ter % te ll Printf to

us e theﬁrs t op erandoverand overagain. Secon d,the # adverb for %o or %x or %X te lls Printf

to emita0 or 0x or 0X preﬁxrespectively.

Rune lit eralsare writt enasacharac ter wit hin singlequotes. ThesimplestexampleisanASCII

ch arac ter like 'a',but it’spossibletowrite any Unico de co de pointeit her direc tly orwit h

numericescap es, as wewill see shortly.

Runesare print edwit h %c,orwit h %q if quoting isdesired:

ascii := 'a'

unicode := 'D'

newline := '\n'

fmt.Printf("%d %[1]c %[1]q\n", ascii) // "97 a 'a'"

fmt.Printf("%d %[1]c %[1]q\n", unicode) // "22269 D 'D'"

fmt.Printf("%d %[1]q\n", newline) // "10 '\n'"

3.2. Floating-PointNumbers

Go provides two sizes of ﬂoating-p ointnumbers, float32 and float64.Their arit hmetic

prop erties aregov erned bythe IEEE 754 stand ard imp lemente d by all moder n CPUs.

Values ofthese numer ic typesrange fro m tiny tohuge. The limits of ﬂoating-p ointvalues can

be found inthe math package. The con stant math.MaxFloat32,the largest float32,isabout

3.4e38,and math.MaxFloat64 is about 1.8e308.The smallestpositive values arenear

1.4e-45 and 4.9e-324,respectively.

A float32 prov ides approximatelysix decimal digitsofpre cision, where asafloat64

prov ides about15dig its; float64 shouldbepreferred for mostpur pos es becaus e float32

comp utation s acc umulateerror rapid lyunlessone isquite caref ul, and the smallestpositive

integerthatcannot beexac tly represent ed as a float32 is not large:

var f float32 = 16777216 // 1 << 24

fmt.Println(f == f+1) // "true"!

Flo at ing-p ointnumbers can bewritt enlit eral lyusingdecimals, like this:

const e = 2.71828 // (approximately)

Dig its may beomitt edbeforethe decimal point (.707)orafter it(1.). Ver y smal l or ver y

large numbers are betterwritt eninscientiﬁcnot ation,wit h theletter e or E precedingthe dec-

imal expon ent:

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

SECTION 3.2. FLOATING-POINT NUMBERS 57

const Avogadro = 6.02214129e23

const Planck =6.62606957e-34

Flo at ing-p ointvalues areconvenientlyprint edwit h Printf’s %g verb,whichcho osesthe most

comp act represent ation thathas adequ ate pre cision, but for tables ofdat a, the %e (exp onent)

or %f (noexp onent)for msmay bemoreappro priate. All three verbs allow ﬁeldwidth and

numericpre cisiontobecontrol led.

for x := 0; x < 8; x++ {

fmt.Printf("x = %d eA =%8.3f\n", x, math.Exp(float64(x)))

}

Thecodeabove printsthe pow ers of e with thre e de cimal digitsofpre cision, aligne d in an

eig ht-charac ter ﬁeld:

x=0 eA =1.000

x=1 eA =2.718

x=2 eA =7.389

x=3 eA =20.086

x=4 eA =54.598

x=5 eA =148.413

x=6 eA =403.429

x=7 eA =1096.633

In addition toalarge col lec tion of the usu almat hemat ical functions,the math packagehas

func tions for cre ating and detec ting the speci al values deﬁne d by IEEE 754: the positive and

negat ive inﬁnities, whichrepresent numb ers ofexcessive mag nitude andthe resultofdiv ision

by zero; andNaN (‘‘notanumber’’), the resultofsuchmat hemat ical lydubious operat ions as

0/0 or Sqrt(-1).

var z float64

fmt.Println(z, -z, 1/z, -1/z, z/z) // "0 -0 +Inf -Inf NaN"

Thefunction math.IsNaN testswhether its argumentisanot-a-numbervalue,and math.NaN

returnssuchavalue.It’stempt ing touse NaN as a sentinel value in a numer ic comp utation,

buttesting whether a speciﬁccomputation alresultisequ altoNaN isfraug htwit h peril

becaus e anycomparisonwit h NaN always yields false:

nan := math.NaN()

fmt.Println(nan == nan, nan < nan, nan > nan) // "false false false"

If a functionthatretur nsaﬂoating-p ointresultmig htfai l,it’sbettertorep ort the fai luresep a-

rately, likethis:

func compute() (value float64, ok bool) {

// ...

if failed {

return 0, false

}

return result, true

}

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

58 CHAPTER 3. BASIC DATATYPES

Thenext program illustrates ﬂoating-p ointgraphics comp utation.Itplots a functionoftwo

var iables z=f(x, y) as a wiremesh3-D sur face,usingScalableVec tor Graphics (SVG), a

st and ard XML not ation for linedraw ings. Figure3.1 shows an exampleofits out put for the

func tion sin(r)/r,where r is sqrt(x*x+y*y).

Figure 3.1. Asur face plot ofthe function sin(r)/r.

gopl.io/ch3/surface

// Surface computes an SVG rendering of a 3-D surface function.

package main

import (

"fmt"

"math"

)

const (

width, height = 600, 320 // canvas size in pixels

cells = 100 // number of grid cells

xyrange = 30.0 // axis ranges (-xyrange..+xyrange)

xyscale = width / 2 / xyrange // pixels per x or y unit

zscale = height * 0.4 // pixels per z unit

angle = math.Pi / 6 // angle of x, y axes (=30°)

)

var sin30, cos30 = math.Sin(angle), math.Cos(angle) // sin(30°), cos(30°)

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

SECTION 3.2. FLOATING-POINT NUMBERS 59

func main() {

fmt.Printf("<svg xmlns='http://www.w3.org/2000/svg' "+

"style='stroke: grey; fill: white; stroke-width: 0.7' "+

"width='%d' height='%d'>", width, height)

for i := 0; i < cells; i++ {

for j := 0; j < cells; j++ {

ax, ay := corner(i+1, j)

bx, by := corner(i, j)

cx, cy := corner(i, j+1)

dx, dy := corner(i+1, j+1)

fmt.Printf("<polygon points='%g,%g %g,%g %g,%g %g,%g'/>\n",

ax, ay, bx, by, cx, cy, dx, dy)

}

fmt.Println("</svg>")

}

func corner(i, j int) (float64, float64) {

// Find point (x,y) at corner of cell (i,j).

x:=xyrange * (float64(i)/cells - 0.5)

y:=xyrange * (float64(j)/cells - 0.5)

// Compute surface height z.

z:=f(x, y)

// Project (x,y,z) isometrically onto 2-D SVG canvas (sx,sy).

sx := width/2 + (x-y)*cos30*xyscale

sy := height/2 + (x+y)*sin30*xyscale - z*zscale

return sx, sy

}

func f(x, y float64) float64 {

r:=math.Hypot(x, y) // distance from (0,0)

return math.Sin(r) / r

}

No tice thatthe function corner returnstwo values, the coordinates ofthe cor ner ofthe cel l.

Theexplanation of how the program wor ksrequires onlybasic geometr y,but it’sﬁne toskip

ov erit, since the point istoillustrate ﬂoating-p ointcomputation.The ess enceofthe program

is mappingbet weenthree dif ferentcoordinatesystems,shown inFigure3.2. The ﬁrs t is a 2-D

gr idof100&100 cellsidentiﬁed byint egercoordinates (i, j), startingat(0, 0) in the far back

corner.Weplotfro m thebacktothe fro ntsothatbackg round polygons may beobs cured by

foreground ones.

Thesecon d co ordinatesystemisameshof3-D ﬂoating-p ointcoordinates (x, y, z), where x

and y arelinearfunctions of i and j,transl ate d so thatthe originisinthe center, and scale d by

thecon stant xyrange.The heig ht z is the value ofthe sur face func tion f (x, y).

Thethirdcoordinatesystemisthe 2-D imagecanvas, wit h (0, 0) in the top lef t corner.Points

in thisplane are denot ed(sx, sy). Weuse anisometr icpro jec tion tomap each3-D point

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

60 CHAPTER 3. BASIC DATATYPES

Figure 3.2. Three dif ferentcoordinatesystems.

(x, y, z)ont o the2-D canvas. A pointapp earsfar thertothe rig htonthe canvas the gre aterits

x value orthe sm all er its y value.And a point app earsfar therdow n thecanvas the gre aterits x

value or y value,and the smaller its z value.The ver tic al andhor izont alscale factors for x and

y areder ive d from the sineand cosineofa30° angle. The scale factorfor z,0.4, isanarbit rar y

parameter.

Fo r each cel l in the 2-D grid, the main functioncomputesthe coordinates onthe image canvas

of the fourcor nersofthe polygon AB CD,where B correspond s to (i, j)and A, C,and D areits

neig hbors,thenprints an SVG ins tructiontodraw it.

Exercis e 3.1: If the function f returnsanon-ﬁnite float64 value,the SVG ﬁle will contain

invalid <polygon> elements (although manySVG renderershandlethisgracef ully). Modif y

theprogram toskipinvalid polygons.

Exercis e 3.2: Experiment wit h visu alizat ions ofother functions fro m the math package. Can

youpro duce an eggbox,mogu ls, orasadd le?

Exercis e 3.3: Coloreachpolygonbas edonits heig ht, sothatthe peaks arecolored red

(#ff0000)and the val leys blue(#0000ff).

Exercis e 3.4: Fo llowing the appro ach ofthe Lissajous exampleinSec tion 1.7, cons truct a web

server thatcomputessur faces andwritesSVG dat a to the client. Theser ver mustset the Con-

tent-Type he ader like this:

w.Header().Set("Content-Type", "image/svg+xml")

(Thisstepwas not required inthe Lissajous examplebecause the ser ver usesstand ard

heur ist ics torecog nize common for mats like PNG fro m theﬁrs t 512 bytes ofthe respons e and

generates the pro per header.) Allow the clienttospecif y values like heig ht, width,and color as

HT TP re questparameters.

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

SECTION 3.3. COMPLEX NUMBERS 61

3.3. ComplexNumbers

Go provides two sizes ofcomplex numbers, complex64 and complex128,whose components

are float32 and float64 resp ectively. The bui lt-in function complex createsacomplex num-

berfro m itsrealand imaginary components, andthe bui lt-in real and imag func tions ext ract

thos e comp onents:

var x complex128 = complex(1, 2) // 1+2i

var y complex128 = complex(3, 4) // 3+4i

fmt.Println(x*y) // "(-5+10i)"

fmt.Println(real(x*y)) // "-5"

fmt.Println(imag(x*y)) // "10"

If a ﬂoating-p ointlit eral ordecimal integerlit eral isimmediate lyfol low edby i,suchas

3.141592i or 2i,itbecom esan im aginary lit era l,denot ing a complex numberwit h azeroreal

comp onent:

fmt.Println(1i * 1i) // "(-1+0i)", i$ =-1

Un der the rules for con stant arithmetic, comp lex cons tants can beadde d to other cons tants

(integerorﬂoating point,realorimaginary), allowing ustowrite complex numbers natural ly,

li ke 1+2i,orequivalently, 2i+1.The declarat ions of x and y ab ove can besimpliﬁed:

x:=1+2i

y:=3+4i

Comp lex numbers may becompare d forequ ality with == and !=.Two complex numbers are

equal if their realpar ts areequ aland their imag inar y partsare equ al.

The math/cmplx packageprovides librar y func tions for wor kingwit h comp lex numbers,such

as the complex squ are root and exp onent iat ionfunctions.

fmt.Println(cmplx.Sqrt(-1)) // "(0+1i)"

Thefol low ing program uses complex128 ar ithmetictogenerateaMandelbro t set.

gopl.io/ch3/mandelbrot

// Mandelbrot emits a PNG image of the Mandelbrot fractal.

package main

import (

"image"

"image/color"

"image/png"

"math/cmplx"

"os"

)

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

62 CHAPTER 3. BASIC DATATYPES

func main() {

const (

xmin, ymin, xmax, ymax = -2, -2, +2, +2

width, height =1024, 1024

)

img := image.NewRGBA(image.Rect(0, 0, width, height))

for py := 0; py < height; py++ {

y:=float64(py)/height*(ymax-ymin) + ymin

for px := 0; px < width; px++ {

x:=float64(px)/width*(xmax-xmin) + xmin

z:=complex(x, y)

// Image point (px, py) represents complex value z.

img.Set(px, py, mandelbrot(z))

}

png.Encode(os.Stdout, img) // NOTE: ignoring errors

}

func mandelbrot(z complex128) color.Color {

const iterations = 200

const contrast = 15

var v complex128

for n := uint8(0); n < iterations; n++ {

v=v*v + z

if cmplx.Abs(v) > 2 {

return color.Gray{255 - contrast*n}

}

return color.Black

}

Thetwo neste d lo ops iterateovereachpoint ina1024&1024 grays cale rasterimage represent-

ingthe −2to+2por tionofthe complex plane.The program tests whether rep eatedlysqu ar-

ingand addingthe numberthatpoint representsevent ual ly ‘‘es cap es’’ thecircleofradius 2.If

so,the point isshade d by the numberofiterat ions ittooktoescap e.Ifnot,the value belon gs

to the Mandelbro t set, andthe point remainsblack.Final ly, the program writestoits stand ard

output the PNG-enco dedimage ofthe iconic frac tal,shown inFigure3.3.

Exercis e 3.5: Implementafull-colorMandelbro t setusingthe function image.NewRGBA and

thetyp e color.RGBA or color.YCbCr.

Exercis e 3.6: Su persamp lingisatechnique toreduce the effec t of pixe lat ionbycomputing the

colorvalue at several pointswit hin eachpixel and tak ingthe average. The simplestmet hod is

to div ide eachpixel int o four ‘‘su bpixe ls.’’ Implementit.

Exercis e 3.7: Anot her simplefrac tal usesNewton’smet hod toﬁnd complex solut ion s to a

func tionsuchas z

−1=0.Shade eachstartingpoint bythe numberofiterat ions required to

getclos e to one ofthe fourroots. Color eachpoint bythe rootitappro ach es.

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

SECTION 3.4. BOOLEANS 63

Figure 3.3. TheMandelbro t set.

Exercis e 3.8: Render ingfrac talsathig h zo omlevel s demands gre atarithmeticpre cision.

Implementthe samefrac tal usingfourdif ferentrepresent ation s of numbers: complex64, com-

plex128, big.Float,and big.Rat.(Thelattertwo typ es arefound inthe math/big package.

Float us esarbit rar y butbounded-pre cision ﬂoating-p oint; Rat us esunb ounde d-pre cision

rat ionalnumbers.) How dothe y comp are inper for mance andmemor y us age?Atwhatzoom

le vel s do render ingartifac ts become visible?

Exercis e 3.9: Wr ite a web ser ver thatrendersfrac talsand writesthe image dat a to the client.

Al low the clienttospecif y the x, y,and zoomvalues as parameterstothe HTTPrequest.

3.4. Booleans

Avalue oftyp e bool,or boolean,has onlytwo possiblevalues, true and false.The con di-

tion s in if and for st atementsare boole ans, andcomparisonoperator s li ke == and < produce

aboole an resu lt. Theunary operator ! is log ical negat ion, so !true is false,or, one mig ht

say, (!true==false)==true,alt hough asamatterofsty le, wealways simplif y re dundant

boole an expressions like x==true to x.

Booleanvalues can becom bine d with the && (AND) and || (OR) operator s,whichhave sh ort-

circuit behavior : if the answerisalready deter mined bythe value ofthe lef t op erand, the rig ht

op erandisnot evaluate d, making itsafetowrite expressions likethis:

s!=""&&s[0] == 'x'

where s[0] wouldpanic if applie d to anemp tystr ing .

Since && hashig her pre cedence than || (mnemonic: && is boole an mu ltiplic ation, || is

boole an addition), noparenthes es arerequired for con dit ion s of thisfor m:

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

64 CHAPTER 3. BASIC DATATYPES

if 'a' <= c && c <= 'z' ||

'A' <= c && c <= 'Z' ||

'0' <= c && c <= '9' {

// ...ASCII letter or digit...

}

Thereisnoimp licitconversionfro m aboole an value toa numer ic value like 0 or1,orvice

vers a.It’snecessary touse anexplicit if, as in

i:=0

if b {

i=1

}

It mig htbewor thwriting a conversionfunction if thisoperat ionwereneeded often:

// btoi returns 1 if b is true and 0 if false.

func btoi(b bool) int {

if b {

return 1

}

return 0

}

Theinv ers e op erat ionissosimplethatitdoesn’twar rantafunction, but for symmetr y here it

is:

// itob reports whether i is non-zero.

func itob(i int) bool { return i != 0 }

3.5. Strings

Astr ing isanimmut ablesequence ofbytes. Str ingsmay cont ain arbit rar y data,including

byteswit h value 0, but usu allythe y cont ain human-readabletext. Text str ingsare convent ion-

al lyint erprete d as UTF-8-enco dedsequences ofUnico de co de points (runes), whichwe’ll

explore indet ailver y so on.

Thebui lt-in len func tionretur nsthe numberofbytes (not runes) in a str ing , andthe in dex

op erat ion s[i] retr ieves the i-t h byte ofstr ing s,where 0 ≤ i < len(s).

s:="hello, world"

fmt.Println(len(s)) // "12"

fmt.Println(s[0], s[7]) // "104 119" ('h' and 'w')

At tempt ing toaccessabyteoutside thisrange results in a panic:

c:=s[len(s)] // panic: index out of range

The i-t h byte ofastr ing isnot necessarily the i-t h ch ara cter of a str ing , becaus e theUTF-8

enco dingofanon-ASCII code point requires two ormorebytes. Wor kingwit h ch arac tersis

discussedshortly.

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

SECTION 3.5. STRINGS 65

The substr ing op erat ion s[i:j] yields a new str ing con sisting ofthe bytes ofthe originalstr ing

st artingatindex i andcontinuingupto, but not including, the byte at index j.The result

cont ains j-i bytes.

fmt.Println(s[0:5]) // "hello"

Again, a panic results if either index isout of bound s or if j is lessthan i.

Either orbot h of the i and j op erands maybeomitt ed, inwhichcas e thedefau ltvalues of 0

(t hestart ofthe str ing)and len(s) (its end) areassume d,respectively.

fmt.Println(s[:5]) // "hello"

fmt.Println(s[7:]) // "world"

fmt.Println(s[:]) // "hello, world"

The + op erator makes a new str ing bycon catenat ingtwo str ings:

fmt.Println("goodbye" + s[5:]) // "goodbye, world"

St rings may becompare d with comparisonoperator s li ke == and <;the comparisonisdon e

byte bybyte, sothe resultisthe natural lexicographic order ing.

St ringvalues areimmut able: the bytesequence cont ained inastr ing value can never be

ch ange d,thoug h of cours e we can assig n anew value toastr ing variab le.Toapp end one

st ringtoanother,for ins tance,wecan write

s:="left foot"

t:=s

s+=",right foot"

Thisdoesnot modif y thestr ing that s or iginallyheldbut causes s to holdthe newstr ing

formed bythe += st atement;meanw hile, t st i l l cont ainsthe old str ing .

fmt.Println(s) // "left foot, right foot"

fmt.Println(t) // "left foot"

Sincestr ingsare immut able, con str uctions thattry tomodif y astr ing’sdat a in place arenot

al lowe d:

s[0] = 'L' // compile error: cannot assign to s[0]

Immu tabilit y me ans thatitissafefor two copies ofastr ing toshare the sameunderly ing

memory,mak ingitche aptocopystr ingsofany lengt h. Simi larly,astr ing s andasubst ring

li ke s[7:] maysafelyshare the samedat a, so the subst ringoperat ionisals o ch eap.Nonew

memory isallocated ineit her cas e.Figure3.4 illustrates the arrangement ofastr ing and two

of itssubst rings sharing the sameunderly ing bytearray.

3.5.1. String Literals

Astr ing value can bewritt en as a st ring litera l,asequence ofbytes enclos ed in dou ble quotes:

"Hello, BF"

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

66 CHAPTER 3. BASIC DATATYPES

Figure 3.4. Thestr ing "hello, world" andtwo subst rings.

Becaus e Go source ﬁles arealways enco dedinUTF-8 andGotext str ingsare convent ion ally

interprete d as UTF-8, wecan include Unico de co de points in str ing lit erals.

Wi thin a double-quote d st ringlit eral, es cap e sequ ences that beg in with a backslash \ canbe

us edtoins ert arbit rar y byte values into the str ing . Oneset ofescap es hand les ASCII cont rol

co des li kenewline, car r iageretur n, andtab:

\a ‘‘aler t’’ or bel l

\b backsp ace

\f form feed

\n ne wline

\r carriageretur n

\t tab

\v vertic al tab

\' singlequote (on ly in the runelit eral '\'')

\" double quote (on lywit hin "..." literals)

\\ backsl ash

Arbit rar y bytescan also beinclude d in literal str ingsusinghexade cimal oroctal escapes. A

hexadecimalescap e is writt en \xhh,wit h exac tly two hexade cimal digits h (in upp erorlow er

case). An oc tal escap e is writt en \ooo with exac tly three octal digits o (0 through 7)not

exce e ding \377.Bot h denot e asinglebytewit h thespeciﬁed value.Later,we’ll see how to

enco de Unico de co de points numer ical ly in str ing lit erals.

A rawstr ing litera l is writt en `...`,usingbackquotesins teadofdou ble quotes. Wit hin a raw

st ringlit eral,noescap e sequences arepro cessed; the contentsare taken literal ly, including

backsl ashesand newlines, soaraw str ing lit eral may spreadoverseveral lines in the program

source.The onlypro cessingisthatcar r iageretur nsare deleted sothatthe value ofthe str ing

is the sameonall platfor ms, includingthose thatconvent ion allyput car r iageretur nsintext

ﬁles.

Rawstr ing lit eralsare a convenientway towrite regu lar expressions,whichtendtohavelotsof

backsl ashes. They are als o us efulfor HTMLtempl ates, JSONlit erals, commandusage mes-

sages, andthe like, whichoften extendovermultiplelines.

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

SECTION 3.5. STRINGS 67

const GoUsage = `Go is a tool for managing Go source code.

Usage:

go command [arguments]

...`

3.5.2. Unicode

Long ago,lifewas simpleand there was, at leastinaparochi al view,onlyone charac ter set to

de al with:ASCII, the AmericanStand ard Codefor Infor mat ionInt erc hange . ASCII, ormore

precis ely US-ASCII, uses7bitstorepresent 128 ‘‘ch arac ters’’:the upp er- and low er-case letters

of Eng lish, digits, andavar ietyofpunctuation and device-cont rol charac ters. For muchofthe

earlydays ofcomputing , this was adequ ate, but it lef t aver y large frac tionofthe wor ld’s

popu lat ionunabletouse their own writ ing systems incomputers.Wit h thegrowt h of the

Internet, dat a in myr iadlangu ages has becom e much morecommon. How can thisrichvar i-

et y be dealt wit h at alland, if possible, efﬁciently?

TheanswerisUnico de (unicode.org), whichcol lec tsall ofthe charac tersinall ofthe wor ld’s

wr iting systems,plu s accents andother diacr iticalmarks, cont rol codes like tab andcar r iage

return,and plenty ofesoterica,and assig nseachone a stand ard numbercal le d a Un ico d e co d e

point or, in Go ter minolog y,arune.

Unico de version8deﬁnescodepointsfor over120,000 charac tersinwel l ov er100 langu ages

andscr ipts. How are these represent edincomputerprogramsand dat a? Thenatural dat a

type toholdasingleruneis int32,and that’swhatGouses; ithas the synonym rune for

precis ely thispur pos e.

We c ou ldrepresent a sequence ofrunes as a sequence of int32 values. Inthisrepresent ation,

whichiscal le d UTF-32 orUCS-4, the encodingofeachUnico de co de pointhas the samesize,

32 bits. Thisissimpleand unifor m,but it usesmuchmorespace thannecessary since most

comp uter-re adabletext isinASCII, whichrequires only8bitsor1byteper charac ter.All the

ch arac tersinwidespreaduse still numberfewer than65,536, whichwou ldﬁtin16bits. Can

we dobetter?

3.5.3. UTF-8

UTF-8 isavar iable-lengt h enco dingofUnico de co de points as bytes. UTF-8 was invent edby

KenThompsonand Rob Pike, two ofthe cre ators ofGo, and isnow a Unico de st and ard . It

us esbet ween1and 4 bytes torepresent eachrune, but only 1 bytefor ASCII charac ters, and

on ly2or3bytes for mostrunes in common use.The hig h-order bitsofthe ﬁrs t byte ofthe

enco dingfor a runeindic atehow manybytes fol low.Ahig h-order 0 indic ates 7-bit ASCII,

whereeachrunetakes only1byte, soitisidenticaltoconvent ion alASCII. A hig h-order 110

indic ates thatthe runetakes 2 bytes; the secon d byte beg inswit h 10.Largerrunes haveanalo-

gous encodings.

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

68 CHAPTER 3. BASIC DATATYPES

0xxxxxx runes0−127 (ASCII)

11xxxxx 10xxxxxx 128−2047 (values <128 unus ed)

110xxxx 10xxxxxx 10xxxxxx 2048−65535 (values <2048 unus ed)

1110xxx 10xxxxxx 10xxxxxx 10xxxxxx 65536−0x10f f f f (other values unus ed)

Avar iable-lengt h enco dingpre cludes direc t indexingtoaccessthe n-t h ch arac ter ofastr ing ,

butUTF-8 has manydesirable pro per ties tocompens ate. The encodingiscompact,compati-

blewit h ASCII, andself-sy nchro nizing: it’spossibletoﬁnd the beg inningofacharac ter by

back ing upnomorethanthree bytes. It’sals o apreﬁxcode, soitcan bedecoded fro m lef t to

rightwit houtany ambiguity orlookahead. Norune’sencodingisasubst ringofany other,or

even ofa sequence ofothers, soyou can searc h forarunebyjustsearc hingfor itsbytes,

withoutwor rying about the pre cedingcontext. Thelexicog raphic byteorder equ als the Uni-

co de co de pointorder,sosor tingUTF-8 wor ksnatural ly. There are noemb edde d NUL (zero)

bytes, whichisconvenientfor prog ramminglangu ages thatuse NUL toter minatestr ings.

Go source ﬁles arealways enco dedinUTF-8, andUTF-8 isthe preferred encodingfor text

st rings manipu lated byGoprograms. The unicode packageprovides functions for wor king

with indiv idu alrunes (suchasdistinguishingletters fro m numb ers,orconvertinganupp er-

case lettertoalow er-case one), andthe unicode/utf8 packageprovides functions for encod-

ingand decodingrunes as bytes usingUTF-8.

Many Unico de ch arac tersare hardtotyp e on a key board ortodistinguish visuallyfro m sim-

ilar-lo oking ones; som e areeveninv isible. Unico de es cap es in Gostr ing lit eralsallow usto

sp ecif y them bytheir numer ic co de pointvalue.There are two for ms, \uhhhh fora16-bit

value and \Uhhhhhhhh fora32-bit value,where each h is a hexade cimal digit;the need for the

32-bit for m ar isesver y inf requently. Eachdenot esthe UTF-8 enco dingofthe speciﬁed code

point. Thus,for example, the fol low ing str ing lit eralsall represent the samesix-bytestr ing:

"BF"

"\xe4\xb8\x96\xe7\x95\x8c"

"\u4e16\u754c"

"\U00004e16\U0000754c"

Thethree escap e sequences abov e prov ide alternat ive not ation s forthe ﬁrs t st ring, but the val-

ues the y denot e areidentical.

Unico de es cap es mayals o be used in runelit erals. Thes e thre e literalsare equivalent:

'B''\u4e16''\U00004e16'

Arunewhose value islessthan256 may bewritt enwit h asinglehexade cimal escape,suchas

'\x41' for 'A',but for hig her values, a \u or \U es cap e mu stbeused. Con sequently,

'\xe4\xb8\x96' is not a legal runelit eral,eventhoug h thos e thre e bytesare a valid UTF-8

enco dingofasinglecodepoint.

Than ks to the nice pro per ties ofUTF-8, manystr ing operat ions don’trequiredecoding. We

cantestwhether one str ing containsanother as a preﬁx:

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

SECTION 3.5. STRINGS 69

func HasPrefix(s, prefix string) bool {

return len(s) >= len(prefix) && s[:len(prefix)] == prefix

}

or asasufﬁx:

func HasSuffix(s, suffix string) bool {

return len(s) >= len(suffix) && s[len(s)-len(suffix):] == suffix

}

or asasubst ring:

func Contains(s, substr string) bool {

for i := 0; i < len(s); i++ {

if HasPrefix(s[i:], substr) {

return true

}

return false

}

usingthe samelog ic forUTF-8-enco dedtext as for raw bytes. Thisisnot truefor other

enco dings. (Thefunctions above are draw n from the strings package, thoug h itsimp lemen-

tation of Contains us esahashingtechnique tosearc h more efﬁciently.)

On the other hand, ifwereallycareabout the indiv idu alUnico de ch arac ters, wehavetouse

ot her mechanisms. Con sider the str ing fro m ourver y ﬁrs t example, whichincludes two East

Asiancharac ters. Figure3.5 illustrates its represent ation inmemor y.The str ing contains13

bytes, but int erprete d as UTF-8, itencodes onlyninecodepointsorrunes:

import "unicode/utf8"

s:="Hello, BF"

fmt.Println(len(s)) // "13"

fmt.Println(utf8.RuneCountInString(s)) // "9"

To pro cessthose charac ters, weneed a UTF-8 deco der.The unicode/utf8 packageprovides

on e that wecan use likethis:

for i := 0; i < len(s); {

r, size := utf8.DecodeRuneInString(s[i:])

fmt.Printf("%d\t%c\n", i, r)

i+=size

}

Each cal l to DecodeRuneInString returns r,the runeits elf,and size,the numberofbytes

occupied bythe UTF-8 enco dingof r.The size isusedtoupdatethe byteindex i of the next

rune inthe str ing . Butthisisclumsy, and weneed loops ofthiskindall the time. For tunately,

Go’s range lo op, whenapp lie d to a str ing , perfor msUTF-8 deco dingimp licitly. The out put of

theloopbelow isals o shown inFigure3.5; not ice how the index jumps bymorethan1for

each non-ASCII rune.

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

70 CHAPTER 3. BASIC DATATYPES

Figure 3.5. A range lo opdecodes a UTF-8-enco dedstr ing .

for i, r := range "Hello, BF"{

fmt.Printf("%d\t%q\t%d\n", i, r, r)

}

We cou lduse a simple range lo optocount the numberofrunes in a str ing , li kethis:

n:=0

for _, _ = range s {

n++

}

As wit h theother for msof range lo op, wecan omitthe var iables wedon’tneed:

n:=0

for range s {

n++

}

Or wecan justcal l utf8.RuneCountInString(s).

We mention edearlier thatitismostlyamatterofconvent ion inGothattext str ingsare int er-

preted asUTF-8-enco dedsequences ofUnico de co de points, but for cor rec t us e of range

lo ops onstr ings, it’smorethanaconvent ion,it’sanecessity.Whathappens ifwerange overa

st ringcontainingarbit rar y binary dat a or,for thatmatter, UTF-8 dat a cont ainingerror s?

Each timeaUTF-8 deco der,whether explicitinacal l to utf8.DecodeRuneInString or

implicitinarange lo op, con sumes an unexp ected inp utbyte, itgenerates a speci al Unico de

re placement chara cter, '\uFFFD',whichisusu allyprint edasawhite question markinside a

bl ack hexagon alordiamond-li keshape (.Whenaprogram encounters thisrunevalue,it’s

of ten a signthatsom e upst reampar t of the systemthatgenerated the str ing dat a hasbeen

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

SECTION 3.5. STRINGS 71

care lessinits tre atmentoftext enco dings.

UTF-8 isexception allyconvenient as an int erc hange for mat but wit hin a program runes may

be moreconvenientbecause the y areofunifor m size andare thu s easi lyindexed inarrays and

slices.

A []rune conv ersionapp lie d to a UTF-8-enco dedstr ing retur nsthe sequence ofUnico de

co de points thatthe str ing encodes:

// "program" in Japanese katakana

s:=">+=@?"

fmt.Printf("% x\n", s) // "e3 83 97 e3 83 ad e3 82 b0 e3 83 a9 e3 83 a0"

r:=[]rune(s)

fmt.Printf("%x\n", r) // "[30d7 30ed 30b0 30e9 30e0]"

(Theverb %xin the ﬁrs t Printf insertsaspace bet weeneachpair ofhex digits.)

If a slice ofrunes isconverted toastr ing , it produces the con catenat ionofthe UTF-8 enco d-

ings ofeachrune:

fmt.Println(string(r)) // ">+=@?"

Conv ertinganint egervalue toastr ing int erprets the int egerasarunevalue,and yieldsthe

UTF-8 represent ation of thatrune:

fmt.Println(string(65)) // "A", not "65"

fmt.Println(string(0x4eac)) // "C"

If the runeisinvalid,the replacementcharac ter issubst itute d:

fmt.Println(string(1234567)) // "("

3.5.4. Strings and ByteSlices

Fo urstand ard packages arepar tic ularlyimp ortantfor manipu lat ingstr ings: bytes, strings,

strconv,and unicode.The strings packageprovides manyfunctions for searc hing, replac-

ing, comparing , tr imming, split ting, and joiningstr ings.

The bytes packagehas similarfunctions for manipu lat ingslices ofbytes, oftyp e []byte,

whichshare som e prop erties wit h st rings. Because str ingsare immut able, bui ldingupstr ings

incrementallycan invo l ve alot ofallocat ionand copying . In suchcas es, it’smoreefﬁcientto

us e the bytes.Buffer type,whichwe’ll show in a mom ent.

The strconv packageprovides functions for convertingboole an, integer, and ﬂoating-p oint

values toand fro m their str ing represent ation s,and functions for quoting and unquoting

st rings.

The unicode packageprovides functions like IsDigit, IsLetter, IsUpper,and IsLower for

cl assif yingrunes. Eachfunctiontakes a singleruneargumentand retur nsaboole an. Conv er-

sionfunctions like ToUpper and ToLower conv ert a rune int o thegiven cas e if itisaletter. All

thes e func tions use the Unico de st and ard categor ies for letters,dig its, andsoon. The strings

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

72 CHAPTER 3. BASIC DATATYPES

packagehas similarfunctions,als o called ToUpper and ToLower,thatretur n anew str ing wit h

thespeciﬁed transfor mat ionapp lie d to eachcharac ter ofthe originalstr ing .

The basename func tionbelow was inspired bythe Unix shel l ut i lit y of the samename. Inour

version, basename(s) removesany preﬁxof s that lookslikeaﬁle systempat h with com-

ponentssep arated byslashes, anditremov esany sufﬁxthatlooks like a ﬁle typ e:

fmt.Println(basename("a/b/c.go")) // "c"

fmt.Println(basename("c.d.go")) // "c.d"

fmt.Println(basename("abc")) // "abc"

The ﬁrs t versionof basename do es al l thewor k withoutthe helpoflibrar ies:

gopl.io/ch3/basename1

// basename removes directory components and a .suffix.

// e.g., a => a, a.go => a, a/b/c.go => c, a/b.c.go => b.c

func basename(s string) string {

// Discard last '/' and everything before.

for i := len(s) - 1; i >= 0; i-- {

if s[i] == '/' {

s=s[i+1:]

break

}

// Preserve everything before last '.'.

for i := len(s) - 1; i >= 0; i-- {

if s[i] == '.' {

s=s[:i]

break

}

return s

}

Asimpler versionusesthe strings.LastIndex librar y func tion:

gopl.io/ch3/basename2

func basename(s string) string {

slash := strings.LastIndex(s, "/") // -1 if "/" not found

s=s[slash+1:]

if dot := strings.LastIndex(s, "."); dot >= 0 {

s=s[:dot]

}

return s

}

The path and path/filepath packages provide a moregeneral set offunctions for manip-

ulat inghierarchic al names. The path packagewor kswit h sl ash-delimite d pathsonany plat-

form.Itshouldn’tbeusedfor ﬁle names, but it isappro priatefor other domains, like the pat h

comp onent ofaURL. Bycontrast, path/filepath manipu lates ﬁle names usingthe rules for

thehostplatfor m,suchas /foo/bar forPOSIX or c:\foo\bar on Micros oft Windows.

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

SECTION 3.5. STRINGS 73

Let’scontinuewit h anot her subst ringexample. The taskistotakeastr ing represent ation of an

integer, suchas "12345",and ins ert commasevery three places, as in "12,345".Thisversion

on lywor ksfor int egers;handling ﬂoating-p ointnumbers islef t as a exercise.

gopl.io/ch3/comma

// comma inserts commas in a non-negative decimal integer string.

func comma(s string) string {

n:=len(s)

if n <= 3 {

return s

}

return comma(s[:n-3]) + "," + s[n-3:]

}

Theargumentto comma is a str ing . If its lengt h is lessthanorequ alto3,nocomma is neces-

sary.Other wis e, comma callsits elf rec ursivelywit h asubst ringcon sisting ofall but the last

thre e ch arac ters, andapp end s acomma and the lastthree charac terstothe resultofthe rec ur-

sive cal l.

Astr ing containsanarray ofbytes that, oncecre ate d, is immut able. Bycontrast, the elements

of a byteslice can befre ely modiﬁed.

St rings can beconverted tobyteslices andbackagain:

s:="abc"

b:=[]byte(s)

s2 := string(b)

Conceptu ally, the []byte(s) conv ersionallocates a new bytearray holdingacopyofthe bytes

of s,and yieldsaslice thatreferences the ent irety ofthatarray.Anopt imizingcompi ler may

be abletoavoid the allocat ionand copying insom e cases, but ingeneral copy ing isrequired to

ensure thatthe bytes of s remain unch ange d even if those of b aresubsequentlymodiﬁed.The

conv ersionfro m byte slice backtostr ing wit h string(b) also makes a copy,toens ure

immut abi lit y of the resulting str ing s2.

To avoid conv ersions and unnecessary memor y al location,manyofthe utilit y func tions inthe

bytes packagedirec tly paral leltheir counterpar ts in the strings package. For example, here

arehalf a dozen functions fro m strings:

func Contains(s, substr string) bool

func Count(s, sep string) int

func Fields(s string) []string

func HasPrefix(s, prefix string) bool

func Index(s, sep string) int

func Join(a []string, sep string) string

andthe cor respondingonesfro m bytes:

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

74 CHAPTER 3. BASIC DATATYPES

func Contains(b, subslice []byte) bool

func Count(s, sep []byte) int

func Fields(s []byte) [][]byte

func HasPrefix(s, prefix []byte) bool

func Index(s, sep []byte) int

func Join(s [][]byte, sep []byte) []byte

Theonlydif ference isthatstr ingshavebeenreplace d by byte slices.

The bytes packageprovides the Buffer type for efﬁcientmanipu lat ionofbyteslices. A

Buffer st artsout emp tybut grows as dat a of typ es li ke string, byte,and []byte arewritt en

to it. Asthe examplebelow shows, a bytes.Buffer var iable requires noinitializat ionbecause

itszerovalue isusable:

gopl.io/ch3/printints

// intsToString is like fmt.Sprintf(values) but adds commas.

func intsToString(values []int) string {

var buf bytes.Buffer

buf.WriteByte('[')

for i, v := range values {

if i > 0 {

buf.WriteString(", ")

}

fmt.Fprintf(&buf, "%d", v)

}

buf.WriteByte(']')

return buf.String()

}

func main() {

fmt.Println(intsToString([]int{1, 2, 3})) // "[1, 2, 3]"

}

Wh enapp endingthe UTF-8 enco dingofanarbit rar y rune toabytes.Buffer,it’sbesttouse

bytes.Buffer’s WriteRune method,but WriteByte is ﬁne for ASCII charac terssuchas '['

and ']'.

The bytes.Buffer type isext remelyversatile, and whenwedis cussint erfaces in Chapt er7,

we’ll see how itmay beusedasareplacementfor a ﬁle whene ver an I/O functionrequires a

sin k forbytes (io.Writer)as Fprintf do es ab ove , or a sourceofbytes (io.Reader).

Exercis e 3.10: Wr ite a non-rec ursiveversionof comma,using bytes.Buffer insteadofstr ing

conc atenation.

Exercis e 3.11: En hance comma so thatitdeals cor rec tly wit h ﬂoating-p ointnumbers and an

opt ion alsig n.

Exercis e 3.12: Wr ite a functionthatrep ortswhether two str ingsare anagramsofeachother,

that is, the y cont ain the sameletters inadif ferentorder.

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

SECTION 3.6. CONSTANTS75

3.5.5. Conversions between Strings and Numbers

In addition toconversions bet weenstr ings, runes, andbytes, it’soften necessary toconvert

betweennumer ic values andtheir str ing represent ation s.Thisisdon e with func tions fro m the

strconv package.

To convert anint egertoastr ing , on e opt ion istouse fmt.Sprintf;another istouse the func-

tion strconv.Itoa (‘‘integertoASCII’’):

x:=123

y:=fmt.Sprintf("%d", x)

fmt.Println(y, strconv.Itoa(x)) // "123 123"

FormatInt and FormatUint canbeusedtofor mat numbers inadif ferentbas e:

fmt.Println(strconv.FormatInt(int64(x), 2)) // "1111011"

The fmt.Printf verbs %b, %d, %u,and %x areoften moreconvenientthan Format func tions,

especi ally ifwewanttoinclude addition alinfor mat ionbesides the number:

s:=fmt.Sprintf("x=%b", x) // "x=1111011"

To parse a str ing represent ing anint eger, use the strconv func tions Atoi or ParseInt,or

ParseUint forunsig ned int egers:

x, err := strconv.Atoi("123") // x is an int

y, err := strconv.ParseInt("123", 10, 64) // base 10, up to 64 bits

Thethirdargumentof ParseInt givesthe size ofthe int egertyp e that the resultmustﬁtint o;

forexample, 16imp lies int16,and the speci al value of0imp lies int.Inany cas e,the typ e of

theresult y is always int64,whichyou can thenconvert toasmaller typ e.

Sometimes fmt.Scanf is usef ulfor parsinginp utthatcon sists oforderly mixtures ofstr ings

andnumbers all onasingleline, but it can beinﬂexible, esp eci ally whenhandlingincomplete

or irregu lar input.

3.6. Constants

Cons tants areexpressions whose value isknown tothe compi ler andwhose evaluation isguar-

ante e d to occ ur at comp ile time, not atrun time. The underly ing typ e of every con stant isa

basic typ e: boole an, st ring, ornumber.

A const de clarat iondeﬁnesnamed values thatlooksyntactic ally likevar iables but whose

value iscon stant,whichpre vents accidental(or nefar ious)changesdur ingprogram exec ution.

Fo r inst ance, a con stant ismoreappro priatethanavar iable for a mat hemat ical cons tantlike

pi,since its value won’tchange:

const pi = 3.14159 // approximately; math.Pi is a better approximation

As wit h var iables, a sequence ofcon stantscan appear in one declarat ion; thiswou ldbe

appropriatefor a gro upofrel ate d values:

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

76 CHAPTER 3. BASIC DATATYPES

const (

e=2.71828182845904523536028747135266249775724709369995957496696763

pi = 3.14159265358979323846264338327950288419716939937510582097494459

)

Many computation s on con stantscan becompletelyevaluate d at comp ile time, reducingthe

work necessary atrun timeand enablingother comp iler opt imizat ions.Error s ordinar ily

detec ted atrun timecan berep orted atcompi letimewhentheir operands are con stants, such

as integerdiv isionbyzero, str ing indexingout of bound s,and any ﬂoating-p ointoperat ion

that wou ldresult in a non-ﬁnite value.

Theresults ofall arithmetic, logic al,and comparisonoperat ions app lie d to con stant operands

arethems elves cons tants, as arethe results ofconversions and cal lstocer tain bui lt-in func-

tion s such as len, cap, real, imag, complex,and unsafe.Sizeof (§13.1).

Sincetheir values areknown tothe compi ler,con stant expressions may appear in typ es, sp ecif-

ic ally asthe lengt h of anarray typ e:

const IPv4Len = 4

// parseIPv4 parses an IPv4 address (d.d.d.d).

func parseIPv4(s string) IP {

var p [IPv4Len]byte

// ...

}

Acon stant declarat ionmay specif y atyp e as wel l as a value,but inthe abs ence ofanexplicit

type,the typ e is infer red fro m theexpressiononthe rig ht-handside.Inthe fol low ing ,

time.Duration is a name d type whose underly ing typ e is int64,and time.Minute is a con-

st ant ofthattyp e.Bot h of the con stantsdeclare d belowthu s have the typ e time.Duration as

we ll, asreveale d by %T:

const noDelay time.Duration = 0

const timeout = 5 * time.Minute

fmt.Printf("%T %[1]v\n", noDelay) // "time.Duration 0"

fmt.Printf("%T %[1]v\n", timeout) // "time.Duration 5m0s

fmt.Printf("%T %[1]v\n", time.Minute) // "time.Duration 1m0s"

Wh enasequence ofcon stantsisdeclare d as a gro up, the rig ht-handside expressionmay be

omit ted for all but the ﬁrs t of the gro up, imp l yingthatthe pre vious expressionand its typ e

shouldbeusedagain. For example:

const (

a=1

c=2

)

fmt.Println(a, b, c, d) // "1 1 2 2"

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

SECTION 3.6. CONSTANTS77

Thisisnot ver y us efulifthe imp licitlycopie d right-handside expressionalways evaluatesto

thesamething. But what if it cou ldvar y? Thisbringsusto iota.

3.6.1. The ConstantGenerator iota

A const de clarat ionmay use the constant generat o r iota,whichisusedtocre ate a sequence

of rel ate d values wit houtspellingout eachone explicitly. Inaconst de clarat ion, the value of

iota begins atzeroand incrementsbyone for eachitem in the sequence.

Here’sanexamplefro m the time package, whichdeﬁnesnamed con stantsoftyp e Weekday for

thedays ofthe week, startingwit h zerofor Sunday.Typ es of thiskindare often cal le d enu-

merati ons,or enums forshort.

type Weekday int

const (

Sunday Weekday = iota

Monday

Tuesday

Wednesday

Thursday

Friday

Saturday

)

Thisdeclares Sunday to be0, Monday to be 1, and soon.

We can use iota in morecomplex expressions too,asinthisexamplefro m the net package

whereeachofthe low est 5 bitsofanunsig ned int egerisgiven a dist inc t name and boole an

interpretation:

type Flags uint

const (

FlagUp Flags = 1 << iota // is up

FlagBroadcast // supports broadcast access capability

FlagLoopback // is a loopback interface

FlagPointToPoint // belongs to a point-to-point link

FlagMulticast // supports multicast access capability

)

As iota increments, eachcon stant isassig ned the value of 1<<iota,whichevaluatestosuc-

cessive pow ers oftwo,eachcor respondingtoasinglebit.Wecan use these con stantswit hin

func tions thattest, set, orcle ar on e or moreofthese bits:

gopl.io/ch3/netflag

func IsUp(v Flags) bool {return v&FlagUp == FlagUp }

func TurnDown(v *Flags) {*v&^= FlagUp }

func SetBroadcast(v *Flags) { *v |= FlagBroadcast }

func IsCast(v Flags) bool {return v&(FlagBroadcast|FlagMulticast) != 0 }

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

78 CHAPTER 3. BASIC DATATYPES

func main() {

var v Flags = FlagMulticast | FlagUp

fmt.Printf("%b %t\n", v, IsUp(v)) // "10001 true"

TurnDown(&v)

fmt.Printf("%b %t\n", v, IsUp(v)) // "10000 false"

SetBroadcast(&v)

fmt.Printf("%b %t\n", v, IsUp(v)) // "10010 false"

fmt.Printf("%b %t\n", v, IsCast(v)) // "10010 true"

}

As a morecomplex exampleof iota,thisdeclarat ionnames the pow ers of1024:

const (

_=1<<(10 * iota)

KiB // 1024

MiB // 1048576

GiB // 1073741824

TiB // 1099511627776 (exceeds 1 << 32)

PiB // 1125899906842624

EiB // 1152921504606846976

ZiB // 1180591620717411303424 (exceeds 1 << 64)

YiB // 1208925819614629174706176

)

The iota me chanism has its limits. For example, it’snot possibletogeneratethe more fami l-

iarpow ers of1000 (KB,MB, and soon) because there isnoexp onent iat ionoperator.

Exercis e 3.13: Wr ite const de clarat ions for KB, MB, upthrough YB as compactly asyou can.

3.6.2. Untyped Constants

Cons tants in Goare a bit unu sual. Alt hough a con stant can haveany ofthe basic dat a types

li ke int or float64,includingnamed basic typ es li ke time.Duration,manycon stantsare

notcommitt edtoapar tic ulartyp e.The compi ler representsthese uncommitt edcon stants

with muchgre aternumer ic precisionthanvalues ofbasic typ es, andarithmeticonthemis

more pre cis e than machinearithmetic; you may assume atleast 256 bitsofpre cision. There

aresix ﬂavor s of these uncommitt edcon stants, cal le d unty ped boole an, untypedint eger,

untypedrune, unt ypedﬂoating-p oint, untypedcomplex, andunt ypedstr ing .

By defer r ingthiscommitment, untypedcon stantsnot onlyret ain their higherpre cisionunt i l

later, but the y canpar ticipate inmanymoreexpressions thancommitt edcon stantswit hout

re quir ingconversions.For example, the values ZiB and YiB in the exampleabove are too big

to store inany int egervar iable,but the y areleg itimate con stantsthatmay beusedinexpres-

sions likethisone:

fmt.Println(YiB/ZiB) // "1024"

As another example, the ﬂoating-p ointcon stant math.Pi maybeusedwhere ver any ﬂoating-

pointorcomplex value isneeded:

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

SECTION 3.6. CONSTANTS79

var x float32 = math.Pi

var y float64 = math.Pi

var z complex128 = math.Pi

If math.Pi hadbeencommitt edtoaspeciﬁctyp e such as float64,the resultwou ldnot beas

precis e,and typ e conv ersions wouldberequired touse itwhenafloat32 or complex128

value iswante d:

const Pi64 float64 = math.Pi

var x float32 = float32(Pi64)

var y float64 = Pi64

var z complex128 = complex128(Pi64)

Fo r literals, synt axdeter mines ﬂavor.The lit erals 0, 0.0, 0i,and '\u0000' al l denot e con-

st antsofthe samevalue but dif ferent ﬂavor s:unt ypedint eger, unt ypedﬂoating-p oint, untyped

comp lex, andunt ypedrune, respectively. Simi larly, true and false areunt ypedboole ansand

st ringlit eralsare unt ypedstr ings.

Re call that / mayrepresent int egerorﬂoating-p ointdiv isiondep endingonits operands.

Cons equently, the choice oflit eral may affe ctthe resultofacon stant div isionexpression:

var f float64 = 212

fmt.Println((f - 32) * 5 / 9) // "100"; (f - 32) * 5 is a float64

fmt.Println(5 / 9 * (f - 32)) // "0"; 5/9 is an untyped integer, 0

fmt.Println(5.0 / 9.0 * (f - 32)) // "100"; 5.0/9.0 is an untyped float

On lycon stantscan beunt yped. When an unt ypedcon stant isassig ned toavar iable, as in the

ﬁrs t st atement below,orapp earsonthe rig ht-handside ofavar iable decl arat ionwit h an

explicittyp e, as in the other three statements, the con stant isimp licitlyconverted tothe typ e

of thatvar iable if possible.

var f float64 = 3 + 0i // untyped complex -> float64

f=2 // untyped integer -> float64

f=1e123 // untyped floating-point -> float64

f='a' // untyped rune -> float64

Thestatementsabove are thu s equivalenttothese:

var f float64 = float64(3 + 0i)

f=float64(2)

f=float64(1e123)

f=float64('a')

Wh ether implicitorexplicit, conv ertingacon stant fro m on e type toanother requires thatthe

targettyp e canrepresent the originalvalue.Roundingisallow edfor realand complex ﬂoat-

ing-p ointnumbers:

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

80 CHAPTER 3. BASIC DATATYPES

const (

deadbeef = 0xdeadbeef // untyped int with value 3735928559

a=uint32(deadbeef) // uint32 with value 3735928559

b=float32(deadbeef) // float32 with value 3735928576 (rounded up)

c=float64(deadbeef) // float64 with value 3735928559 (exact)

d=int32(deadbeef) // compile error: constant overflows int32

e=float64(1e309) // compile error: constant overflows float64

f=uint(-1) // compile error: constant underflows uint

)

In a var iable decl arat ionwit hout an explicittyp e (includingshort var iable decl arat ions), the

ﬂavor of the unt ypedcon stant imp licitlydeter mines the defau lttyp e of the var iable,asinthese

examples:

i:=0 //untyped integer; implicit int(0)

r:='\000'// untyped rune; implicit rune('\000')

f:=0.0 // untyped floating-point; implicit float64(0.0)

c:=0i//untyped complex; implicit complex128(0i)

No tethe asy mmet ry: unt ypedint egers are converted to int,whose size isnot guarante e d, but

untypedﬂoating-p ointand complex numbers are converted tothe explicitlysize d types

float64 and complex128.The langu agehas nounsize d float and complex typesanalogou s

to unsize d int,because itisver y difﬁcult towrite cor rec t numericalalgor it hms wit hout

know ing the size ofone’sﬂoating-p ointdat a types.

To givethe var iable a different typ e,wemustexplicitlyconvert the unt ypedcon stant tothe

desired typ e or state the desired typ e in the var iable decl arat ion, as in these examples:

var i = int8(0)

var i int8 = 0

Thes e defau lts arepar tic ularlyimp ortantwhenconvertinganunt ypedcon stant toanint erface

value (see Chapter 7) since the y deter mineits dynamic typ e.

fmt.Printf("%T\n", 0) // "int"

fmt.Printf("%T\n", 0.0) // "float64"

fmt.Printf("%T\n", 0i) // "complex128"

fmt.Printf("%T\n", '\000')//"int32" (rune)

We’ve now cov ere d thebasic dat a typesofGo. The next stepistoshowhow the y canbecom-

bine d into largergro upingslikearrays andstr ucts, andthenint o data str uctures for solv ing

re alprogrammingpro blems; thatisthe topic ofChapt er4.

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

CompositeTypes

In Chapt er3 wedis cussedthe basic typ es that ser veasbui ldingblo cks for dat a st ruc tures in a

Go program; the y arethe atoms ofour univers e. In thischapt er, we’ll takealookat comp osite

types, the mole cules create d by com biningthe basic typ es in various ways. We’ll tal k ab out

four suchtyp es—ar rays, slices, maps, andstr ucts—andatthe end ofthe chapt er, we’ll show

howstr uctured dat a usingthese typ es canbeencoded asand parsedfro m JSONdat a andused

to generateHTMLfro m temp lates.

Ar rays andstr uctsare ag gre gat e types; their values arecon catenat ions ofother values in mem-

or y.Arrays arehom ogene ous—their elements allhavethe sametyp e—whereasstr uctsare

heterogene ous.Bot h ar rays andstr uctsare ﬁxe d size.Incontrast, slices andmaps are

dy namic dat a st ruc tures thatgrow as values areadde d.

4.1. Arrays

An array isaﬁxe d-lengt h sequence ofzeroormoreelements ofapar tic ulartyp e.Because of

their ﬁxe d lengt h, ar rays arerarelyuseddirec tly inGo. Slices, whichcan growand shrink,are

much moreversatile, but tounderst and slices wemustunderst and arrays ﬁrs t.

In div idu alarray elements areaccessedwit h theconvent ion alsubscript not ation,where

su bscriptsrun fro m zerotoone lessthanthe array lengt h. Thebui lt-in function len returns

thenumberofelements in the array.

var a [3]int // array of 3 integers

fmt.Println(a[0]) // print the first element

fmt.Println(a[len(a)-1]) // print the last element, a[2]

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

82 CHAPTER 4. COMPOSITE TYPES

// Print the indices and elements.

for i, v := range a {

fmt.Printf("%d %d\n", i, v)

}

// Print the elements only.

for _, v := range a {

fmt.Printf("%d\n", v)

}

By defau lt, the elements ofanew array variableare initial lyset tothe zerovalue for the ele-

ment typ e,whichis 0 fornumbers.Wecan use an ar ray lit era l to initialize an array wit h alist

of values:

var q [3]int = [3]int{1, 2, 3}

var r [3]int = [3]int{1, 2}

fmt.Println(r[2]) // "0"

In anarray literal,ifanellipsis ‘‘...’’ appearsinplace ofthe lengt h, thearray lengt h is deter-

mined bythe numberofinitializers. Thedeﬁnition of q canbesimpliﬁed to

q:=[...]int{1, 2, 3}

fmt.Printf("%T\n", q) // "[3]int"

Thesize of an array ispar t of itstyp e,so [3]int and [4]int aredif ferenttyp es. Thesize

mu stbeacon stant expression, thatis, an expressionwhose value can becompute d as the

prog ram is beingcompi led.

q:=[3]int{1, 2, 3}

q=[4]int{1, 2, 3, 4} // compile error: cannot assign [4]int to [3]int

As we’ll see,the lit eral synt axissimi lar for arrays, slices, maps, andstr ucts. Thespeciﬁcfor m

ab ove isalistofvalues in order,but it isals o possible tospecif y alistofindex andvalue pairs,

li kethis:

type Currency int

const (

USD Currency = iota

EUR

GBP

RMB

)

symbol := [...]string{USD: "$", EUR: "9", GBP: "!", RMB: """}

fmt.Println(RMB, symbol[RMB]) // "3 ""

In thisfor m,indices can appear in anyorder andsom e maybeomitt ed; as before, uns peciﬁed

values takeonthe zerovalue for the elementtyp e.For ins tance,

r:=[...]int{99: -1}

deﬁnes an array r with 100 elements, allzeroexceptfor the last, whichhas value −1.

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

SECTION 4.1. ARRAYS83

If anarray’selementtyp e is comp arab le then the array typ e is comparable too,sowemay

direc tly compare two arrays ofthattyp e usingthe == op erator,whichrep ortswhether allcor-

resp ondingelements areequ al. The != op erator isits negat ion.

a:=[2]int{1, 2}

b:=[...]int{1, 2}

c:=[2]int{1, 3}

fmt.Println(a == b, a == c, b == c) // "true false false"

d:=[3]int{1, 2}

fmt.Println(a == d) // compile error: cannot compare [2]int == [3]int

As a moreplausible example, the function Sum256 in the crypto/sha256 packagepro duces

theSHA256 cryptographic hashor di gest of a message store d in an arbit rar y byte slice.The

digesthas 256 bits, soits typ e is [32]byte.Iftwo digests arethe same, itisext remelylikely

that the two messagesare the same; if the digests differ, the two messagesare dif ferent. This

prog ram pr intsand comparesthe SHA256 digests of "x" and "X":

gopl.io/ch4/sha256

import "crypto/sha256"

func main() {

c1 := sha256.Sum256([]byte("x"))

c2 := sha256.Sum256([]byte("X"))

fmt.Printf("%x\n%x\n%t\n%T\n", c1, c2, c1 == c2, c1)

// Output:

// 2d711642b726b04401627ca9fbac32f5c8530fb1903cc4db02258717921a4881

// 4b68ab3847feda7d6c62c1fbcbeebfa35eab7351ed5e78f4ddadea5df64b8015

// false

// [32]uint8

}

Thetwo inp uts differbyonlyasinglebit,but approximatelyhalf the bitsare dif ferentinthe

digests. Not ice the Printf verbs: %x to print all the elements ofanarray orslice ofbytes in

hexade cimal, %t to showaboole an, and %T to displ aythe typ e of a value.

Wh enafunctioniscal le d,acopyofeachargumentvalue isassig ned tothe cor responding

parameter variable, sothe functionreceivesacopy, not the original. Passinglarge arrays in

this way can beinefﬁcient, andany changesthatthe functionmakes toarray elements affe ct

on lythe copy, not the original. Inthisregard, Gotre ats arrays like any other typ e,but this

behavior isdif ferentfro m languagesthatimp licitlypassarrays by reference.

Of course,wecan explicitlypassapoint ertoanarray sothatany modiﬁcation s thefunction

makestoarray elements will bevisible tothe cal ler.Thisfunctionzeroesthe contentsofa

[32]byte ar ray:

func zero(ptr *[32]byte) {

for i := range ptr {

ptr[i] = 0

}

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

84 CHAPTER 4. COMPOSITE TYPES

Thearray literal [32]byte{} yields anarray of32bytes. Eachelementofthe array has the

zerovalue for byte,whichiszero. Wecan use thatfac t to write a dif ferentversionof zero:

func zero(ptr *[32]byte) {

*ptr = [32]byte{}

}

Usingapoint ertoanarray isefﬁcientand allowsthe cal le d func tiontomut ate the cal ler’s

var iable,but arrays arestill inherentlyinﬂexiblebecause oftheir ﬁxe d size.The zero func tion

wi l l notacceptapoint ertoa[16]byte var iable,for example, nor isthere any way toadd or

remove array elements. For these reasons,other thanspeci al caseslikeSHA256’sﬁxe d-size

hash,arrays areseldomused as functionparameters; instead, weuse slices.

Exercis e 4.1: Wr ite a functionthatcountsthe numberofbitsthatare dif ferent in two SHA256

hashes. (See PopCount from Sec tion 2.6.2.)

Exercis e 4.2: Wr ite a program thatprintsthe SHA256 hashofits stand ard inp utbydefau ltbut

supp ortsacommand-line ﬂag toprint the SHA384 orSHA512 hashins tead.

4.2. Slices

Slices represent var iable-lengt h sequences whose elements allhavethe sametyp e.Aslice typ e

is writt en []T,where the elements havetyp e T;itlooks like anarray typ e withoutasize.

Ar rays andslices areint imate lyconne cte d. Aslice isalig htweig htdat a st ruc turethatgives

accesstoasubsequence (orperhaps all) ofthe elements ofanarray,whichisknown asthe

slice’s un derly ing array.Aslice has three components: a point er, a lengt h, andacap acity.The

pointerpointstothe ﬁrs t elementofthe array thatisreach ablethrough the slice,whichisnot

ne cessarily the array’sﬁrs t element. Thelengt h is the numberofslice elements; itcan’texceed

thecap acity,whichisusu allythe numberofelements bet weenthe start ofthe slice andthe end

of the underly ing array.The bui lt-in functions len and cap return thos e values.

Mu ltipleslices can share the sameunderly ing array andmay refer tooverl appingpar ts of that

ar ray.Figure4.1 shows an array ofstr ingsfor the mont hsofthe year, and two overl apping

slices ofit. Thearray isdeclare d as

months := [...]string{1: "January", /* ... */, 12: "December"}

so January is months[1] andDecemberis months[12].Ordinar ily,the array elementatindex

0wou ldcontain the ﬁrs t value,but because monthsare always numbere d from 1,wecan leave

it out of the declarat ionand itwill beinitialize d to anemp tystr ing .

The sli ceoperator s[i:j],where 0 ≤ i ≤ j ≤ cap(s),cre atesanew slice thatreferstoelements

i thro ugh j-1 of the sequence s,whichmay beanarray variable, a point erto an array,or

anot her slice.The resulting slice has j-i elements. If i is omitt ed, it’s0,and if j is omitt ed, it’s

len(s).Thu s theslice months[1:13] refers to the whole range ofvalid months, as does the

slice months[1:];the slice months[:] refers to the whole array.Let’sdeﬁne overl appingslices

forthe secon d qu arter andthe northern summer :

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

SECTION 4.2. SLICES 85

Figure 4.1. Tw o ov erl appingslices ofanarray ofmonths.

Q2 := months[4:7]

summer := months[6:9]

fmt.Println(Q2) // ["April" "May" "June"]

fmt.Println(summer) // ["June" "July" "August"]

June isinclude d in eachand isthe soleout put of this(inefﬁcient) testfor commonelements:

for _, s := range summer {

for _, q := range Q2 {

if s == q {

fmt.Printf("%s appears in both\n", s)

}

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

86 CHAPTER 4. COMPOSITE TYPES

Slicingbeyon d cap(s) caus esapanic, but slicingbeyon d len(s) extends the slice,sothe

resu ltmay belon g erthanthe original:

fmt.Println(summer[:20]) // panic: out of range

endlessSummer := summer[:5] // extend a slice (within capacity)

fmt.Println(endlessSummer) // "[June July August September October]"

As anaside,not e thesimi lar ity ofthe subst ringoperat iononstr ingstothe slice operator on

[]byte slices. Bot h arewritt en x[m:n],and bot h return a subsequence ofthe originalbytes,

sh aring the underly ing represent ation sothatbot h op erat ions takecon stant time. The expres-

sion x[m:n] yields a str ing if x is a str ing , or a []byte if x is a []byte.

Sinceaslice cont ainsapoint ertoanelementofanarray,passingaslice toafunctionper mits

thefunctiontomodif y theunderly ing array elements. Inother words,copying a slice creates

an alias (§2.3.2) for the underly ing array.The function reverse re versesthe elements ofan

[]int slice in place,and itmay beapp lie d to slices ofany lengt h.

gopl.io/ch4/rev

// reverse reverses a slice of ints in place.

func reverse(s []int) {

for i, j := 0, len(s)-1; i < j; i, j = i+1, j-1 {

s[i], s[j] = s[j], s[i]

}

Here wereverse the whole array a:

a:=[...]int{0, 1, 2, 3, 4, 5}

reverse(a[:])

fmt.Println(a) // "[5 4 3 2 1 0]"

Asimpleway to ro tat e aslice leftby n elements istoapp l y the reverse func tionthree times,

ﬁrs t to the leading n elements, thentothe remainingelements, and ﬁnallytothe whole slice.

(Torot ate to the rig ht, makethe thirdcal l ﬁrs t.)

s:=[]int{0, 1, 2, 3, 4, 5}

// Rotate s left by two positions.

reverse(s[:2])

reverse(s[2:])

reverse(s)

fmt.Println(s) // "[2 3 4 5 0 1]"

No tice how the expressionthatinitializes the slice s dif fersfro m that for the array a.Asli ce

litera l lo oks like anarray literal,asequence ofvalues sep arated bycommasand sur rounded by

braces, but the size isnot given. Thisimp licitlycre atesanarray variableofthe rig htsize and

yields a slice thatpointstoit. Aswit h ar ray literals, slice literalsmay specif y thevalues in

order,orgivetheir indices explicitly, oruse a mix ofthe two sty les.

Un likearrays, slices arenot comparable,sowecannot use == to test whether two slices cont ain

thesameelements. Thestand ard librar y prov ides the hig hly opt imize d bytes.Equal func tion

forcomparing two slices ofbytes ([]byte), but for other typ es of slice,wemustdothe

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

SECTION 4.2. SLICES 87

comp arisonourselves:

func equal(x, y []string) bool {

if len(x) != len(y) {

return false

}

for i := range x {

if x[i] != y[i] {

return false

}

return true

}

Gi venhow natural this ‘‘de ep’’ equality testis, andthatitisnomorecostlyatrun timethanthe

== op erator for arrays ofstr ings, itmay bepuzzlingthatslice comp arisons donot als o work

this way.There are two reasons why deepequivalence ispro blemat ic. First,unlikearray ele-

ments, the elements ofaslice areindirec t,mak ingitpossiblefor a slice tocontain its elf.

Although there are ways todealwit h such cas es, none issimple, efﬁcient, andmost

importantly, obv iou s.

Second,because slice elements areindirec t,aﬁxe d slice value may cont ain different elements

at different times as the contentsofthe underly ing array aremodiﬁed.Because a hashtable

such asGo’smap typ e makesonlyshallow copies ofits keys, itrequires thatequ ality for each

ke y remain the samethroughoutthe lifet imeofthe hashtable.Deepequivalence wou ldthu s

make slices unsuit ablefor use asmap keys. For reference typ es li kepoint ers and channel s,the

== op erator tests reference ident ity,thatis, whether the two ent ities refer tothe samething. An

analogou s ‘‘sh allow’’ equality testfor slices couldbeuseful, and itwou ldsolvethe pro blem

with maps, but the incon sistent tre atmentofslices andarrays bythe == op erator wou ldbe

conf using. The safestchoice istodis allow slice comp arisons alt ogether.

Theonlylegal slice comp arisonisagainst nil, as in

if summer == nil { /* ... */ }

Thezerovalue ofaslice typ e is nil.Ani l slice has nounderly ing array.The nil slice has

lengt h andcap acity zero, but there are als o non-ni l slices oflengt h andcap acity zero, suchas

[]int{} or make([]int, 3)[3:].Aswit h anytyp e that can havenil values, the nil value ofa

partic ularslice typ e canbewritt enusingaconversionexpressionsuchas []int(nil).

var s []int // len(s) == 0, s == nil

s=nil // len(s) == 0, s == nil

s=[]int(nil) // len(s) == 0, s == nil

s=[]int{} // len(s) == 0, s != nil

So,ifyou need totestwhether a slice isemp ty, use len(s) == 0,not s==nil.Other than

comp aring equ alto nil,anil slice beh aveslikeany other zero-lengt h slice; reverse(nil) is

perfec tly safe, for example. Unlesscle arly documente d to the contrar y,Gofunctions should

treatall zero-lengt h slices the sameway,whether nilornon-ni l.

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

88 CHAPTER 4. COMPOSITE TYPES

Thebui lt-in function make createsaslice ofaspeciﬁed elementtyp e,lengt h, andcap acity.The

capacity argumentmay beomitt ed, inwhichcas e thecap acity equ als the lengt h.

make([]T, len)

make([]T, len, cap) // same as make([]T, cap)[:len]

Un der the hood, make createsanunnamed array variableand retur nsaslice ofit; the array is

accessibleonlythrough the retur ned slice.Inthe ﬁrs t form,the slice isavie w of the ent ire

ar ray.Inthe secon d,the slice isavie w of only the array’sﬁrs t len elements, but itscap acity

includes the ent ire array.The addition alelements areset aside for fut ure growt h.

4.2.1. The append Function

Thebui lt-in append func tionapp end s it ems toslices:

var runes []rune

for _, r := range "Hello, BF"{

runes = append(runes, r)

}

fmt.Printf("%q\n", runes) // "['H''e''l''l''o'',''''B''F']"

Theloopuses append to bui ld theslice ofninerunes enco dedbythe str ing lit eral,alt hough

this speciﬁcpro blem ismoreconvenientlysolved byusingthe bui lt-in conv ersion

[]rune("Hello, BF").

The append func tioniscruci al to underst andinghow slices wor k,solet’stakealookatwhatis

goingon. Here’saversioncal le d appendInt that isspeci alizedfor []int slices:

gopl.io/ch4/append

func appendInt(x []int, y int) []int {

var z []int

zlen := len(x) + 1

if zlen <= cap(x) {

// There is room to grow. Extend the slice.

z=x[:zlen]

}else {

// There is insufficient space. Allocate a new array.

// Grow by doubling, for amortized linear complexity.

zcap := zlen

if zcap < 2*len(x) {

zcap = 2 * len(x)

}

z=make([]int, zlen, zcap)

copy(z, x) // a built-in function; see text

}

z[len(x)] = y

return z

}

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

SECTION 4.2. SLICES 89

Each cal l to appendInt mu stche ckwhether the slice has sufﬁcientcap acity toholdthe new

elements in the exist ing array.Ifso, itextends the slice bydeﬁningalargerslice (still wit hin

theoriginalarray), copies the element y into the newspace,and retur nsthe slice.The inp ut x

andthe result z sh are the sameunderly ing array.

If there isins ufﬁcientspace for growt h, appendInt mu stallocateanew array big enough to

hold the result, copy the values fro m x into it, thenapp end the newelement y.The result z

nowreferstoadif ferentunderly ing array thanthe array that x refers to.

It wou ldbestraig htfor wardtocopythe elements wit h explicitloops, but it’seasier touse the

built-in function copy,whichcopies elements fro m on e slice toanother ofthe sametyp e.Its

ﬁrs t argumentisthe destination and its secon d is the source, res emblingthe order ofoperands

in an assig nmentlike dst = src.The slices may refer tothe sameunderly ing array ; they may

even overl ap. Alt hough wedon’tuse ithere, copy returnsthe numberofelements actu ally

copied,whichisthe smaller ofthe two slice lengt hs, sothere isnodangerofrunningoff the

endoroverwriting som ethingout of range .

Fo r efﬁciency,the newarray isusu allysom ewhat largerthanthe minimum needed tohold x

and y.Exp andingthe array bydou blingits size at eachexp ansionavoidsanexcessive number

of allocat ions and ens uresthatapp endingasingleelementtakes cons tanttimeonaverage.

Thisprogram demon strates the effec t:

func main() {

var x, y []int

for i := 0; i < 10; i++ {

y=appendInt(x, i)

fmt.Printf("%d cap=%d\t%v\n", i, cap(y), y)

x=y

}

Each change incap acity indic ates an allo cat ionand a copy:

0cap=1 [0]

1cap=2 [0 1]

2cap=4 [0 12]

3cap=4 [0 123]

4cap=8 [0 1234]

5cap=8 [0 12345]

6cap=8 [0 123456]

7cap=8 [0 1234567]

8cap=16 [0 12345678]

9cap=16 [0 123456789]

Let’stakeaclos er lo okatthe i=3 it erat ion. Theslice x cont ainsthe three elements [0 1 2] but

hascap acity 4,sothere isasingleelementofslack atthe end,and appendInt of the element 3

maypro ceed wit houtreallocat ing. The resulting slice y haslengt h andcap acity 4,and has the

same underly ing array as the originalslice x, as Figure4.2 shows.

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

90 CHAPTER 4. COMPOSITE TYPES

Figure 4.2. Ap pendingwit h ro omtogrow.

On the next iterat ion, i=4,there isnoslack atall,so appendInt al locatesanew array ofsize 8,

copies the fourelements [0 1 2 3] of x,and app end s 4,the value of i.The resulting slice y

hasalengt h of 5 but a cap acity of8;the slackof3will savethe next three iterat ions fro m the

ne e d to reallocate. The slices y and x arevie ws of dif ferentarrays. Thisoperat ionisdepic ted

in Figure4.3.

Figure 4.3. Ap pendingwit houtroomtogrow.

Thebui lt-in append func tionmay use a moresop histicate d growth strateg y than appendInt’s

simplisticone.Usu allywedon’tknowwhether a given cal l to append wi l l caus e areallocat ion,

so wecan’tassume thatthe originalslice referstothe samearray as the resulting slice,nor that

it referstoadif ferentone.Simi larly,wemustnot assume thatoperat ions onelements ofthe

oldslice will (or will not)bereﬂec ted inthe newslice.Asaresult, it’susu altoassig n theresult

of a cal l to append to the sameslice variablewhose value wepassedto append:

runes = append(runes, r)

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

SECTION 4.2. SLICES 91

Up dat ingthe slice variableisrequired not justwhencal ling append,but for any functionthat

maychange the lengt h or cap acity ofaslice ormakeitrefer toadif ferentunderly ing array.To

us e slices correc tly,it’simp ortanttobearinmindthatalt hough the elements ofthe underly ing

ar ray areindirec t,the slice’spoint er, lengt h, andcap acity are not.Toupdatethemrequires an

assig nmentlikethe one above . In thisrespect,slices arenot ‘‘pure’’ referencetyp es butres em-

ble an aggregatetyp e such asthisstr uct:

type IntSlice struct {

ptr *int

len, cap int

}

Our appendInt func tionaddsasingleelementtoaslice,but the bui lt-in append lets usadd

more thanone newelement, orevenawhole slice ofthem.

var x []int

x=append(x, 1)

x=append(x, 2, 3)

x=append(x, 4, 5, 6)

x=append(x, x...) // append the slice x

fmt.Println(x) // "[1 2 3 4 5 6 1 2 3 4 5 6]"

Wi ththe small modiﬁcation shown below,wecan match the beh avior of the bui lt-in append.

Theellipsis ‘‘...’’ in the declarat ionof appendInt makesthe function variadic:itaccepts any

numb erofﬁnalarguments. Thecor respondingellipsisinthe cal l ab ove to append shows how

to sup ply a listofarguments fro m aslice.We’ll explain thismechanism in detai l in

Section5.7.

func appendInt(x []int, y ...int) []int {

var z []int

zlen := len(x) + len(y)

// ...expand z to at least zlen...

copy(z[len(x):], y)

return z

}

Thelog ic to exp and z’sunderly ing array remainsunchange d andisnot shown.

4.2.2. In-PlaceSliceTechniques

Let’ssee moreexamples offunctions that, like rotate and reverse,modif y theelements ofa

slice in place.Given a list ofstr ings, the nonempty func tionretur nsthe non-empt y on es:

gopl.io/ch4/nonempty

// Nonempty is an example of an in-place slice algorithm.

package main

import "fmt"

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

92 CHAPTER 4. COMPOSITE TYPES

// nonempty returns a slice holding only the non-empty strings.

// The underlying array is modified during the call.

func nonempty(strings []string) []string {

i:=0

for _, s := range strings {

if s != "" {

strings[i] = s

i++

}

return strings[:i]

}

Thesubtlepar t is thatthe inp utslice andthe out put slice share the sameunderly ing array.

Thisavoidsthe need toallocateanother array,thoug h of cours e thecontentsof data arepar tly

ov erwritt en, as evidence d by the secon d pr int statement:

data := []string{"one", "", "three"}

fmt.Printf("%q\n", nonempty(data)) // `["one" "three"]`

fmt.Printf("%q\n", data) // `["one" "three" "three"]`

Thus wewou ldusu allywrite: data = nonempty(data).

The nonempty func tioncan also bewritt enusing append:

func nonempty2(strings []string) []string {

out := strings[:0] // zero-length slice of original

for _, s := range strings {

if s != "" {

out = append(out, s)

}

return out

}

Whiche ver variant weuse,reusinganarray in thisway requires thatatmostone out put value

is pro duce d foreachinp utvalue,whichistrueofmanyalgor it hms thatﬁlt erout elements ofa

sequence orcom bine adj acentones. Suchint ric ateslice usage isthe exception,not the rule,

butitcan becle ar,efﬁcient, andusefulonocc asion.

Aslice can beusedtoimp lementastack.Given an initial lyemp tyslice stack,wecan pusha

ne w value ont o theend ofthe slice wit h append:

stack = append(stack, v) // push v

Thetop ofthe stack isthe lastelement:

top := stack[len(stack)-1] // top of stack

andshr inking the stack bypoppingthatelementis

stack = stack[:len(stack)-1] // pop

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

SECTION 4.3. MAPS 93

To remov e an elementfro m themidd le of a slice,preservingthe order ofthe remainingele-

ments, use copy to slide the hig her-numbere d elements down byone toﬁll the gap:

func remove(slice []int, i int) []int {

copy(slice[i:], slice[i+1:])

return slice[:len(slice)-1]

}

func main() {

s:=[]int{5, 6, 7, 8, 9}

fmt.Println(remove(s, 2)) // "[5 6 8 9]"

}

And if we don’tneed topreservethe order,wecan justmov e thelastelementint o thegap:

func remove(slice []int, i int) []int {

slice[i] = slice[len(slice)-1]

return slice[:len(slice)-1]

}

func main() {

s:=[]int{5, 6, 7, 8, 9}

fmt.Println(remove(s, 2)) // "[5 6 9 8]

}

Exercis e 4.3: Re writ e reverse to use anarray point erins teadofaslice.

Exercis e 4.4: Wr ite a versionof rotate that operates in a singlepass.

Exercis e 4.5: Wr ite anin-place functiontoeliminateadj acentdup lic ates in a []string slice.

Exercis e 4.6: Wr ite anin-place functionthatsqu asheseachrun ofadj acentUnico de sp aces

(s ee unicode.IsSpace) in a UTF-8-enco ded []byte slice into a singleASCII space.

Exercis e 4.7: Mo dif y reverse to reverse the charac tersofa[]byte slice thatrepresentsa

UTF-8-enco dedstr ing , in place.Can you doitwit houtallocat ingnew memor y?

4.3. Maps

Thehashtable isone ofthe most ingenious and versatileofall dat a st ruc tures. Itisan

unordered col lec tion of key/value pairsinwhichall the keysare distinc t,and the value asso ci-

ated wit h agiven key can beret rie ved,updated,orremov edusingacon stant numb erofkey

comp arisons onthe average, nomatterhow large the hashtable.

In Go, a map is a reference toahashtable,and a map typ e is writt en map[K]V,where K and V

arethe typ es of itskeysand values. All ofthe keys in a given map areofthe sametyp e,and all

of the values areofthe sametyp e,but the keysneed not beofthe sametyp e as the values. The

ke y type K mu stbecomparable using ==,sothatthe map can testwhether a given key isequ al

to one already wit hin it. Though ﬂoating-p ointnumbers are comparable,it’sabad ideato

comp are ﬂoats for equ ality and,aswemention edinChapt er3,esp eci ally bad if NaN isapos-

siblevalue.There are norestr ictions onthe value typ e V.

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

94 CHAPTER 4. COMPOSITE TYPES

Thebui lt-in function make canbeusedtocre ate a map:

ages := make(map[string]int) // mapping from strings to ints

We can also use a maplit era l to cre ate a new map popu lated wit h some initial key/value pairs:

ages := map[string]int{

"alice": 31,

"charlie": 34,

}

Thisisequivalentto

ages := make(map[string]int)

ages["alice"] = 31

ages["charlie"] = 34

so analt ernat ive expressionfor a new emp tymap is map[string]int{}.

Mapelements areaccessedthrough the usu alsubscript not ation:

ages["alice"] = 32

fmt.Println(ages["alice"]) // "32"

andremov edwit h thebui lt-in function delete:

delete(ages, "alice") // remove element ages["alice"]

Al l of these operat ions are safeevenifthe elementisn’tinthe map; a map lookup usingakey

that isn’tpresent retur nsthe zerovalue for itstyp e,so, for ins tance,the fol low ing workseven

when "bob" is not yet a key inthe map because the value of ages["bob"] wi l l be 0.

ages["bob"] = ages["bob"] + 1 // happy birthday!

Theshorthand assig nmentfor ms x+=yand x++ also wor k formap elements, sowecan re-

wr ite the statement above as

ages["bob"] += 1

or evenmorecon cis ely as

ages["bob"]++

Butamap elementisnot a var iable,and wecannot takeits address:

_=&ages["bob"] // compile error: cannot take address of map element

Onereasonthatwecan’ttakethe addressofamap elementisthatgrowing a map might cause

re hashingofexist ing elements into new storagelocat ions,thu s potent ial lyinvalid atingthe

address.

To e numerateall the key/value pairsinthe map,weuse a range-b ased for lo opsimi lar to

thos e we saw for slices. Successive iterat ions ofthe loopcause the name and age var iables to

be set tothe next key/value pair :

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

SECTION 4.3. MAPS 95

for name, age := range ages {

fmt.Printf("%s\t%d\n", name, age)

}

Theorder ofmap iterat ionisuns peciﬁed,and dif ferentimp lementation s mig htuse a dif ferent

hash function, leadingtoadif ferentorder ing. Inprac tice,the order israndom, varying fro m

on e exec ution tothe next. Thisisint ent ion al; mak ingthe sequence varyhelps force programs

to berobustacrossimp lementation s.Toenumeratethe key/value pairsinorder,wemustsor t

thekeysexplicitly, for ins tance,usingthe Strings func tionfro m the sort package if the keys

arestr ings. Thisisacommonpattern:

import "sort"

var names []string

for name := range ages {

names = append(names, name)

}

sort.Strings(names)

for _, name := range names {

fmt.Printf("%s\t%d\n", name, ages[name])

}

Sinceweknowthe ﬁnalsize of names from the outset, itismoreefﬁcienttoallocateanarray of

therequired size upfro nt. Thestatement below cre atesaslice thatisinitial lyemp tybut has

sufﬁcientcap acity toholdall the keysofthe ages map:

names := make([]string, 0, len(ages))

In the ﬁrs t range lo opabove , we requireonlythe keysofthe ages map, soweomitthe secon d

lo opvar iable.Inthe secon d lo op, werequireonlythe elements ofthe names slice,soweuse

theblank identiﬁer _ to ignorethe ﬁrs t var iable,the index.

Thezerovalue for a map typ e is nil,thatis, a reference tonohashtable at all.

var ages map[string]int

fmt.Println(ages == nil) // "true"

fmt.Println(len(ages) == 0) // "true"

Most operat ions onmaps, includinglooku p, delete, len,and range lo ops, aresafetoper-

form onanil map reference,since itbeh aveslikeanemp tymap.But storing toanil map

caus esapanic:

ages["carol"] = 21 // panic: assignment to entry in nil map

Yo u mu stallocatethe map beforeyou can store int o it.

Accessingamap elementbysubscript ing always yieldsavalue.Ifthe key ispresent inthe

map, you get the cor respondingvalue; if not,you get the zerovalue for the elementtyp e,aswe

sawwit h ages["bob"].For manypur pos es that’sﬁne,but som etimesyou need toknow

whet her the elementwas reallythere ornot.For example, ifthe elementtyp e is numer ic, you

mig hthavetodistinguish bet weenanon existentelementand anelementthathappens tohave

thevalue zero, usingatestlikethis:

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

96 CHAPTER 4. COMPOSITE TYPES

age, ok := ages["bob"]

if !ok { /* "bob" is not a key in this map; age == 0. */ }

Yo u ’ll often see these two statementscom bine d,likethis:

if age, ok := ages["bob"]; !ok { /* ... */ }

Su bscript ing a map in thiscontext yieldstwo values; the secon d is a boole an that rep orts

whet her the elementwas present.The boole an var iable isoften cal le d ok,esp eci ally ifitis

immediate lyused in an if condit ion.

As wit h slices, maps cannot becompare d to eachother ; theonlylegal comp arisoniswit h nil.

To testwhether two maps cont ain the samekeysand the sameass oci ated values, wemust

wr ite a loop:

func equal(x, y map[string]int) bool {

if len(x) != len(y) {

return false

}

for k, xv := range x {

if yv, ok := y[k]; !ok || yv != xv {

return false

}

return true

}

Observehow weuse !ok to distinguish the ‘‘missing’’ and ‘‘pres entbut zero’’ cases. Had we

naïvelywritt en xv != y[k],the cal l belowwou ldincor rec tly rep ort its arguments as equ al:

// True if equal is written incorrectly.

equal(map[string]int{"A": 0}, map[string]int{"B": 42})

Go doesnot provide a set type,but since the keysofamap aredistinc t,amap can ser vethis

purpos e.Toillustrate, the program dedup re ads a sequence oflines andprintsonlythe ﬁrs t

occurrence ofeachdistinc t line. (It’savar iantofthe dup prog ram that weshowe d in

Section1.3.) The dedup prog ram us esamap whose keysrepresent the set oflines thathave

already app eared toens ure thatsubsequentocc urrences arenot print ed.

gopl.io/ch4/dedup

func main() {

seen := make(map[string]bool) // a set of strings

input := bufio.NewScanner(os.Stdin)

for input.Scan() {

line := input.Text()

if !seen[line] {

seen[line] = true

fmt.Println(line)

}

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

SECTION 4.3. MAPS 97

if err := input.Err(); err != nil {

fmt.Fprintf(os.Stderr, "dedup: %v\n", err)

os.Exit(1)

}

Go programmersoften descr ibe a map usedinthisfashionasa‘‘setofstr ings’’ withoutfur ther

ado,but beware, not all map[string]bool values aresimplesets; som e maycontain bot h true

and false values.

Sometimesweneed a map orset whose keysare slices, but because a map’skeysmustbecom-

parable,thiscannot beexpress eddirec tly.How ever, itcan bedon e in two steps. First we

deﬁne a helperfunction k that maps eachkey toastr ing , with thepro per tythat k(x) == k(y)

if andonlyifwecon sider x and y equivalent. Then wecre ate a map whose keysare str ings,

apply ing the helperfunctiontoeachkey beforeweaccessthe map.

Theexamplebelow usesamap torecordthe numberoftimes Add hasbeencal le d with a given

list ofstr ings. Ituses fmt.Sprintf to convert a slice ofstr ingsint o asinglestr ing thatisa

suit ablemap key,quoting eachslice elementwit h %q to recordstr ing bound aries faithfully:

var m = make(map[string]int)

func k(list []string) string { return fmt.Sprintf("%q", list) }

func Add(list []string) {m[k(list)]++ }

func Count(list []string) int { return m[k(list)] }

Thesameappro ach can beusedfor any non-comp arable key typ e,not justslices. It’seven

us efulfor comparable key typ es when you wantadeﬁnition of equ ality other than ==,suchas

case-insensit ive comparisons for str ings. And the typ e of k(x) ne e dn’tbeastr ing;any com-

parable typ e with thedesired equivalence pro per tywill do, such as int egers,arrays, orstr ucts.

Here’sanother exampleofmaps in action,aprogram thatcountsthe occ urrences ofeachdis-

tinc t Unico de co de point in its input. Since there are a large numberofpossiblecharac ters,

on lyasmall frac tionofwhichwou ldapp ear in anypar tic ulardocument, a map isanatural

way tokeeptrackofjustthe onesthathavebeenseenand their correspondingcounts.

gopl.io/ch4/charcount

// Charcount computes counts of Unicode characters.

package main

import (

"bufio"

"fmt"

"io"

"os"

"unicode"

"unicode/utf8"

)

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

98 CHAPTER 4. COMPOSITE TYPES

func main() {

counts := make(map[rune]int) // counts of Unicode characters

var utflen [utf8.UTFMax + 1]int // count of lengths of UTF-8 encodings

invalid := 0 // count of invalid UTF-8 characters

in := bufio.NewReader(os.Stdin)

for {

r, n, err := in.ReadRune() // returns rune, nbytes, error

if err == io.EOF {

break

}

if err != nil {

fmt.Fprintf(os.Stderr, "charcount: %v\n", err)

os.Exit(1)

}

if r == unicode.ReplacementChar && n == 1 {

invalid++

continue

}

counts[r]++

utflen[n]++

}

fmt.Printf("rune\tcount\n")

for c, n := range counts {

fmt.Printf("%q\t%d\n", c, n)

}

fmt.Print("\nlen\tcount\n")

for i, n := range utflen {

if i > 0 {

fmt.Printf("%d\t%d\n", i, n)

}

if invalid > 0 {

fmt.Printf("\n%d invalid UTF-8 characters\n", invalid)

}

The ReadRune method perfor msUTF-8 deco dingand retur nsthree values: the decoded rune,

thelengt h in bytes ofits UTF-8 enco ding, and anerror value.The onlyerror weexp ect isend-

of-ﬁle.Ifthe inp utwas not a legal UTF-8 enco dingofarune, the retur ned runeis uni-

code.ReplacementChar andthe lengt h is 1.

The charcount prog ram also printsacount ofthe lengt hsofthe UTF-8 enco dings ofthe

runesthatapp eared inthe inp ut. A mapisnot the bestdat a st ruc turefor that; since enco ding

lengt hsrange onlyfro m 1to utf8.UTFMax (w hichhas the value 4), an array ismorecompact.

As anexp eriment, weran charcount on thisbookits elf at one point.Alt hough it’smostlyin

English, ofcours e, it doeshaveafair numberofnon-ASCII charac ters. Hereare the top ten:

°27 B 15 F 14 é 13 A 10 < 5 & 5 D 4 ( 4 + 3

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

SECTION 4.4. STRUCTS99

andhereisthe distr ibution of the lengt hsofall the UTF-8 enco dings:

len count

1765391

260

370

Thevalue typ e of a map can its elf beacomposite typ e,suchasamap orslice.Inthe fol low ing

co de,the key typ e of graph is string andthe value typ e is map[string]bool,represent ing a

setofstr ings. Con ceptu ally, graph maps a str ing toaset ofrel ate d st rings, its successors ina

direc ted graph.

gopl.io/ch4/graph

var graph = make(map[string]map[string]bool)

func addEdge(from, to string) {

edges := graph[from]

if edges == nil {

edges = make(map[string]bool)

graph[from] = edges

}

edges[to] = true

}

func hasEdge(from, to string) bool {

return graph[from][to]

}

The addEdge func tionshows the idiomaticway topopu lateamap lazi ly, thatis, toinitialize

each value as its key app earsfor the ﬁrs t time.The hasEdge func tionshows how the zero

value ofamissingmap entr y is often put towor k:evenifneither from nor to is present,

graph[from][to] wi l l always giveameaningf ul resu lt.

Exercis e 4.8: Mo dif y charcount to count letters,dig its, andsoonintheir Unico de categories,

usingfunctions like unicode.IsLetter.

Exercis e 4.9: Wr ite a program wordfreq to rep ort the fre quency ofeachword in an inp uttext

ﬁle.Cal l input.Split(bufio.ScanWords) before the ﬁrs t call to Scan to bre akthe inp utint o

word s insteadoflines.

4.4. Structs

A st ruc t is anaggregatedat a type thatgro ups toget her zeroormorenamed values ofarbit rar y

typesasa singleent ity.Eachvalue iscal le d a ﬁel d.The classic exampleofastr uct fro m data

processingisthe emp loy eerecord, whose ﬁelds are a unique ID,the emp loy ee’sname, address,

date ofbir th, position,sal ary, manager,and the like. All ofthese ﬁelds are col lec ted int o asin-

gleent ity thatcan becopie d as a unit, passedtofunctions and retur ned bythem, store d in

ar rays, andsoon.

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

100 CHAPTER 4. COMPOSITE TYPES

Thes e twostatementsdeclare a str uct typ e called Employee andavar iable cal le d dilbert that

is anins tance ofan Employee:

type Employee struct {

ID int

Name string

Address string

DoB time.Time

Position string

Salary int

ManagerID int

}

var dilbert Employee

Theindiv idu alﬁelds of dilbert areaccessedusingdot not ation like dilbert.Name and

dilbert.DoB.Because dilbert is a var iable,its ﬁelds are var iables too,sowemay assig n to a

ﬁeld:

dilbert.Salary -= 5000 // demoted, for writing too few lines of code

or takeits addressand accessitthrough a point er:

position := &dilbert.Position

*position = "Senior " + *position // promoted, for outsourcing to Elbonia

Thedot not ation als o workswit h apoint ertoastr uct:

var employeeOfTheMonth *Employee = &dilbert

employeeOfTheMonth.Position += " (proactive team player)"

Thelaststatement isequivalentto

(*employeeOfTheMonth).Position += " (proactive team player)"

Gi venanemp loy ee’sunique ID,the function EmployeeByID returnsapoint ertoan Employee

st ruc t. We can use the dot not ation toaccessits ﬁelds:

func EmployeeByID(id int) *Employee { /* ... */ }

fmt.Println(EmployeeByID(dilbert.ManagerID).Position) // "Pointy-haired boss"

id := dilbert.ID

EmployeeByID(id).Salary = 0 // fired for... no real reason

Thelaststatement updates the Employee st ruc t that ispoint edtobythe resultofthe cal l to

EmployeeByID.Ifthe resulttyp e of EmployeeByID were change d to Employee insteadof

*Employee,the assig nmentstatement wou ldnot compi lesince its left-handside wou ldnot

identify a var iable.

Fieldsare usu allywritt enone per line, wit h theﬁeld’snamepre cedingits typ e,but con sec utive

ﬁelds ofthe sametyp e maybecom bine d, as wit h Name and Address here:

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

SECTION 4.4. STRUCTS101

type Employee struct {

ID int

Name, Address string

DoB time.Time

Position string

Salary int

ManagerID int

}

Field order issig niﬁcant totyp e identity.Had weals o combined the declarat ionofthe Posi-

tion ﬁeld(also a str ing), orint erc hange d Name and Address,wewou ldbedeﬁningadif ferent

st ruc t type.Typic ally weonlycom bine the declarat ions ofrel ate d ﬁelds.

Thenameofastr uct ﬁeldisexp orted ifitbeg inswit h acapit alletter; thisisGo’smain access

cont rol mechanism. A st ruc t type may cont ain a mixtureofexp orted and unexp orted ﬁelds.

St ruc t typestendtobeverb osebecause the y of ten invo l ve alinefor each ﬁeld. Alt hough we

couldwrite out the whole typ e each timeitisneeded,the rep etit ion wou ldget tires ome.

In ste ad, str uct typ es usuallyapp ear wit hin the declarat ionofanamed typ e li ke Employee.

Anamed str uct typ e S can’tdeclare a ﬁeldofthe sametyp e S:anaggregatevalue cannot con-

tain its elf.(An analogou s rest ric tionapp lies toarrays.) But S maydeclare a ﬁeldofthe

pointertyp e *S,whichlets uscre ate rec ursivedat a st ruc tures like lin ked lists andtre es. Thisis

illustrated inthe codebelow,whichusesabinar y tree toimp lement an ins ertionsor t:

gopl.io/ch4/treesort

type tree struct {

value int

left, right *tree

}

// Sort sorts values in place.

func Sort(values []int) {

var root *tree

for _, v := range values {

root = add(root, v)

}

appendValues(values[:0], root)

}

// appendValues appends the elements of t to values in order

// and returns the resulting slice.

func appendValues(values []int, t *tree) []int {

if t != nil {

values = appendValues(values, t.left)

values = append(values, t.value)

values = appendValues(values, t.right)

}

return values

}

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

102 CHAPTER 4. COMPOSITE TYPES

func add(t *tree, value int) *tree {

if t == nil {

// Equivalent to return &tree{value: value}.

t=new(tree)

t.value = value

return t

}

if value < t.value {

t.left = add(t.left, value)

}else {

t.right = add(t.right, value)

}

return t

}

Thezerovalue for a str uct iscomposedofthe zerovalues ofeachofits ﬁelds.Itisusu ally

desirable thatthe zerovalue beanatural orsensibledefau lt. For example, in bytes.Buffer,

theinitial value ofthe str uct isaready-to-use emp tybuf fer,and the zerovalue of sync.Mutex,

whichwe’ll see inChapt er9,isaready-to-use unlocke d mu tex. Som etimesthissensibleinitial

behavior happens for fre e,but som etimesthe typ e desig ner has towor k at it.

Thestr uct typ e with noﬁelds is cal le d the emptystr uct,writt en struct{}.Ithas size zeroand

carries noinfor mat ionbut may beusefulnon etheless. Som e Go programmersuse itins tead

of bool as the value typ e of a map thatrepresentsaset, toemp hasize thatonlythe keysare sig-

niﬁcant,but the space sav ingismarginaland the syntaxmorecumbers ome,sowegeneral ly

avoidit.

seen := make(map[string]struct{}) // set of strings

// ...

if _, ok := seen[s]; !ok {

seen[s] = struct{}{}

// ...first time seeing s...

}

4.4.1. Struct Literals

Avalue ofastr uct typ e canbewritt enusingast ruc t litera l that speciﬁes values for itsﬁelds.

type Point struct{ X, Y int }

p:=Point{1, 2}

Thereare two for msofstr uct lit eral.The ﬁrs t form,shown above , re quires thatavalue be

sp eciﬁed for ever y ﬁeld, inthe rig htorder.Itburdensthe writer(andreader) wit h rememb er-

ingexac tly whatthe ﬁelds are , anditmakes the codefrag i le shouldthe set of ﬁelds later grow

or bereordered.Accordingly, thisfor m tend s to beusedonlywit hin the packagethatdeﬁnes

thestr uct typ e,orwit h smal ler st ruc t typesfor whichthere is an obv iou s ﬁeldorder ingcon-

vent ion,like image.Point{x, y} or color.RGBA{red, green, blue, alpha}.

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

SECTION 4.4. STRUCTS103

Mo reoften, the secon d form isused, inwhichastr uct value isinitialize d by listing som e or all

of the ﬁeldnames andtheir correspondingvalues, as in thisstatement fro m theLissajous

prog ram of Sec tion 1.4:

anim := gif.GIF{LoopCount: nframes}

If a ﬁeldisomitt edinthiskindoflit eral,itisset tothe zerovalue for itstyp e.Because names

areprovide d,the order of ﬁelds doesn’tmatter.

Thetwo for mscannot bemixed inthe samelit eral.Nor can you use the (order-b ased)ﬁrs t

form oflit eral tosne akaro und the rulethatunexp orted identiﬁersmay not berefer red to

from another package.

package p

type T struct{ a, b int } // a and b are not exported

package q

import "p"

var _ = p.T{a: 1, b: 2} // compile error: can'treference a, b

var _ = p.T{1, 2} // compile error: can'treference a, b

Although the lastlineabove doesn’tmention the unexp orted ﬁeldidentiﬁers, it’sreallyusing

them implicitly, soit’snot allow ed.

St ruc t values can bepassedasarguments tofunctions and retur ned fro m them. For ins tance,

this functionscales a Point by a speciﬁed fac tor :

func Scale(p Point, factor int) Point {

return Point{p.X * factor, p.Y * factor}

}

fmt.Println(Scale(Point{1, 2}, 5)) // "{5 10}"

Fo r efﬁciency,largerstr uct typ es areusu allypassedtoorretur ned fro m func tions indirec tly

usingapoint er,

func Bonus(e *Employee, percent int) int {

return e.Salary * percent / 100

}

andthisisrequired ifthe functionmustmodif y itsargument, since in a cal l-by-value langu age

li keGo, the cal le d func tionreceivesonlyacopyofanargument, not a reference tothe original

argument.

func AwardAnnualRaise(e *Employee) {

e.Salary = e.Salary * 105 / 100

}

Becaus e st ruc ts aresocommonlydealt wit h thro ugh point ers,it’spossibletouse this

shorthand notation tocre ate and initialize a struct var iable andobt ain its address:

pp := &Point{1, 2}

It isexac tly equivalentto

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

104 CHAPTER 4. COMPOSITE TYPES

pp := new(Point)

*pp = Point{1, 2}

but &Point{1, 2} canbeuseddirec tly wit hin an expression, such as a functioncal l.

4.4.2. Comparing Structs

If all the ﬁelds ofa str uct are comparable,the str uct its elf iscomparable,sotwo expressions of

that typ e maybecompare d using == or !=.The == op erat ioncomparesthe cor responding

ﬁelds ofthe two str ucts in order,sothe two print edexpressions below are equivalent:

type Point struct{ X, Y int }

p:=Point{1, 2}

q:=Point{2, 1}

fmt.Println(p.X == q.X && p.Y == q.Y) // "false"

fmt.Println(p == q) // "false"

Comp arable str uct typ es, li keother comp arable typ es, maybeused as the key typ e of a map.

type address struct {

hostname string

port int

}

hits := make(map[address]int)

hits[address{"golang.org", 443}]++

4.4.3. Struct Embedding and Anonymous Fields

In thissec tion,we’ll see how Go’sunu sual st ruc t embedding me chanism lets ususe one named

st ruc t type asan an ony mou s ﬁel d of another str uct typ e,providingaconvenientsyntactic

shortcutsothatasimpledot expressionlike x.f canstand for a chain of ﬁelds like x.d.e.f.

Consider a 2-D drawingprogram thatprovides a librar y of shapes, suchasrec tangles, ellipses,

st ars,and whe els. Hereare two ofthe typ es it mig htdeﬁne:

type Circle struct {

X, Y, Radius int

}

type Wheel struct {

X, Y, Radius, Spokes int

}

A Circle hasﬁelds for the X and Y co ordinates ofits center, and a Radius.AWheel hasall the

fe aturesofaCircle,plu s Spokes,the numberofins cribedradi al sp okes. Let’scre ate a whe el:

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

SECTION 4.4. STRUCTS105

var w Wheel

w.X = 8

w.Y = 8

w.Radius = 5

w.Spokes = 20

As the set ofshapesgrows, we’re bound tonot ice similarities andrep etit ion among them, soit

maybeconvenienttofac tor out their common par ts:

type Point struct {

X, Y int

}

type Circle struct {

Center Point

Radius int

}

type Wheel struct {

Circle Circle

Spokes int

}

Theapp lic ationmay becle arer for it,but thischange makes accessingthe ﬁelds ofaWheel

more verb ose:

var w Wheel

w.Circle.Center.X = 8

w.Circle.Center.Y = 8

w.Circle.Radius = 5

w.Spokes = 20

Go lets usdeclare a ﬁeldwit h atyp e butnoname; such ﬁelds are cal le d an ony mou s ﬁel ds.The

type ofthe ﬁeldmustbeanamed typ e or a point ertoanamed typ e.Below, Circle and Wheel

have one anony mou s ﬁeldeach. Wesay thataPoint is embedded within Circle,and a

Circle is emb edde d within Wheel.

type Circle struct {

Point

Radius int

}

type Wheel struct {

Circle

Spokes int

}

Than ks to emb edding, wecan refer tothe names at the leavesofthe imp licittre e without

giving the int erveningnames:

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

106 CHAPTER 4. COMPOSITE TYPES

var w Wheel

w.X = 8 // equivalent to w.Circle.Point.X = 8

w.Y = 8 // equivalent to w.Circle.Point.Y = 8

w.Radius = 5 // equivalent to w.Circle.Radius = 5

w.Spokes = 20

Theexplicitfor msshown inthe commentsabove are still valid,how ever, showing that

‘‘anonymou s ﬁeld’’ is som ethingofamisnomer. The ﬁelds Circle and Point do have

names—that ofthe named typ e—butthose names areopt ion alindot expressions.Wemay

omit any orall ofthe anony mou s ﬁelds whenselec ting their subﬁelds.

Unfortunately, there’snocor respondingshorthand for the str uct lit eral synt ax, soneither of

thes e wi l l comp ile:

w=Wheel{8, 8, 5, 20} // compile error: unknown fields

w=Wheel{X: 8, Y: 8, Radius: 5, Spokes: 20} // compile error: unknown fields

Thestr uct lit eral mustfol low the shap e of the typ e de clarat ion, sowemustuse one ofthe two

formsbelow,whichare equivalenttoeachother :

gopl.io/ch4/embed

w=Wheel{Circle{Point{8, 8}, 5}, 20}

w=Wheel{

Circle: Circle{

Point: Point{X: 8, Y: 8},

Radius: 5,

Spokes: 20, // NOTE: trailing comma necessary here (and at Radius)

}

fmt.Printf("%#v\n", w)

// Output:

// Wheel{Circle:Circle{Point:Point{X:8, Y:8}, Radius:5}, Spokes:20}

w.X = 42

fmt.Printf("%#v\n", w)

// Output:

// Wheel{Circle:Circle{Point:Point{X:42, Y:8}, Radius:5}, Spokes:20}

No tice how the # adverb causes Printf’s %v verb todispl ayvalues in a for m simi lar toGosyn-

tax. For str uct values, thisfor m includes the nameofeach ﬁeld.

Becaus e ‘‘anonymou s’’ ﬁelds dohaveimp licitnames, you can’thavetwo anony mou s ﬁelds of

thesametyp e since their names wou ldconﬂic t. Andbecause the nameofthe ﬁeldisimp lic-

it lydeter mined byits typ e,sotoo isthe visibilit y of the ﬁeld. Inthe examples abov e,the Point

and Circle anonymou s ﬁelds are exp orted.Had the y been unexp orted (point and circle),

we cou ldstill use the shorthand for m

w.X = 8 // equivalent to w.circle.point.X = 8

butthe explicitlon g form shown inthe comment wou ldbeforbidden outside the declaring

packagebecause circle and point wouldbeinaccessible.

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

SECTION 4.5. JSON 107

Wh atwe’ve seensofar ofstr uct emb eddingisjustaspr ink lingofsyntactic sugar onthe dot

notation usedtoselec t st ruc t ﬁelds.Later,we’ll see thatanony mou s ﬁelds need not bestr uct

types; anynamed typ e or point ertoanamed typ e wi l l do.But why wou ldyou wanttoemb ed

atyp e that has nosubﬁelds?

Theanswerhas to do wit h methods. Theshorthand notation usedfor selec ting the ﬁelds ofan

embedde d type wor ksfor selec ting its met hodsaswel l.Ineffec t,the out erstr uct typ e gains

notjustthe ﬁelds ofthe emb edde d type but itsmet hodstoo.Thismechanism isthe main way

that comp lex obj e ctbeh avior s arecomposedfro m simpler ones. Comp ositi on is central to

objec t-oriente d prog ramming in Go, and we’ll explore itfur ther in Sec tion 6.3.

4.5. JSON

JavaS crip t Objec t No tat ion(JSON) isastand ard not ation for sendingand receiving str uctured

infor mat ion. JSONisnot the onlysuchnot ation.XML (§7.14), ASN.1, andGoogle’sPro tocol

Buffersser vesimi lar pur pos es andeachhas its niche,but because ofits simplicity,readabi lit y,

andunivers alsup por t,JSONisthe most widely used.

Go has excellentsup por t forencodingand decodingthese for mats, provide d by the stand ard

librar y packages encoding/json, encoding/xml, encoding/asn1,and soon, andthese pack-

ages allhavesimi lar APIs.Thissec tion gives a brief overvie w of the most imp ortantpar ts of

the encoding/json package.

JSONisanencodingofJavaS crip t values—st rings, numbers,boole ans, arrays, andobj e cts—as

Unico de text. It’sanefﬁcientyet readablerepresent ation for the basic dat a typesofChapt er3

andthe composite typ es of thischapt er—ar rays, slices, str ucts, andmaps.

Thebasic JSONtyp es arenumbers (in decimal orscientiﬁcnot ation), boole ans(true or

false), andstr ings, whichare sequences ofUnico de co de points enclos edindou ble quotes,

with backslash escap es usingasimi lar not ation toGo, thoug h JSON’s \Uhhhh numericescap es

denot e UTF-16 codes, not runes.

Thes e basic typ es maybecom bine d re cursive lyusingJSONarrays andobj e cts. A JSONarray

is anordered sequence ofvalues, writt enasa comma-s eparated listenclos edinsqu are brack-

ets; JSONarrays areusedtoencodeGoarrays andslices. A JSONobj e ctisamapping fro m

st rings tovalues, writt enasasequence of name:value pairssep arated bycommasand sur-

ro unded bybraces; JSONobj e cts areusedtoencodeGomaps (wit h st ringkeys) andstr ucts.

Fo r example:

boolean true

number -273.15

string "She said \"Hello, BF\""

array ["gold", "silver", "bronze"]

object {"year": 1980,

"event": "archery",

"medals": ["gold", "silver", "bronze"]}

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

108 CHAPTER 4. COMPOSITE TYPES

Consider an applic ationthatgat hersmov ierevie ws andoffersrecommend ation s.Its Movie

data typ e andatypic al list ofvalues aredeclare d below. (Thestr ing lit eralsafter the Year and

Color ﬁelddeclarat ions are ﬁel d tags;we’ll explain them in a mom ent.)

gopl.io/ch4/movie

type Movie struct {

Title string

Year int `json:"released"`

Color bool `json:"color,omitempty"`

Actors []string

}

var movies = []Movie{

{Title: "Casablanca", Year: 1942, Color: false,

Actors: []string{"Humphrey Bogart", "Ingrid Bergman"}},

{Title: "Cool Hand Luke", Year: 1967, Color: true,

Actors: []string{"Paul Newman"}},

{Title: "Bullitt", Year: 1968, Color: true,

Actors: []string{"Steve McQueen", "Jacqueline Bisset"}},

// ...

}

Data str uctures like thisare anexcel lent ﬁtfor JSON, andit’seasy toconvert inbot h

direc tion s.ConvertingaGodat a st ruc turelike movies to JSONiscal le d marshaling.Mar-

sh alingisdon e by json.Marshal:

data, err := json.Marshal(movies)

if err != nil {

log.Fatalf("JSON marshaling failed: %s", err)

}

fmt.Printf("%s\n", data)

Marshal produces a byteslice cont ainingaver y long str ing wit h no ext raneous white space;

we’ve folde d thelines soit ﬁts:

[{"Title":"Casablanca","released":1942,"Actors":["Humphrey Bogart","Ingr

id Bergman"]},{"Title":"Cool Hand Luke","released":1967,"color":true,"Ac

tors":["Paul Newman"]},{"Title":"Bullitt","released":1968,"color":true,"

Actors":["Steve McQueen","Jacqueline Bisset"]}]

Thiscompact represent ation containsall the infor mat ionbut it’shardtoread. For human

cons ump tion, a variant cal le d json.MarshalIndent produces neatlyindente d output.Two

addition alarguments deﬁne a preﬁxfor eachlineofout put and a str ing for eachlevel of

indentation:

data, err := json.MarshalIndent(movies, "", " ")

if err != nil {

log.Fatalf("JSON marshaling failed: %s", err)

}

fmt.Printf("%s\n", data)

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

SECTION 4.5. JSON 109

Thecodeabove prints

[

{

"Title": "Casablanca",

"released": 1942,

"Actors": [

"Humphrey Bogart",

"Ingrid Bergman"

]

{

"Title": "Cool Hand Luke",

"released": 1967,

"color": true,

"Actors": [

"Paul Newman"

]

{

"Title": "Bullitt",

"released": 1968,

"color": true,

"Actors": [

"Steve McQueen",

"Jacqueline Bisset"

]

}

]

Mars halingusesthe Gostr uct ﬁeldnames as the ﬁeldnames for the JSONobj e cts (through

reﬂec tion,aswe’ll see inSec tion 12.6). Only exp orted ﬁelds are marsh ale d,whichiswhy we

ch ose capit alize d namesfor all the Goﬁeldnames.

Yo u mayhavenot ice d that the nameofthe Year ﬁeldchange d to released in the out put,and

Color ch ange d to color.That’sbecause ofthe ﬁel d tags.Aﬁeldtag isastr ing ofmet adat a

asso ciate d at comp ile timewit h theﬁeldofastr uct:

Year int `json:"released"`

Color bool `json:"color,omitempty"`

Aﬁeldtag may beany lit eral str ing , butitisconvent ion allyint erprete d as a space-s eparated

list of key:"value" pairs; since the y cont ain double quotation marks, ﬁeldtags areusu ally

wr itt enwit h raw str ing lit erals. The json ke y cont rol s thebeh avior of the encoding/json

package, and other encoding/... packages fol low thisconvent ion.The ﬁrs t part ofthe json

ﬁeldtag speciﬁes an alternat ive JSONnamefor the Goﬁeld. Field tags areoften usedto

sp ecif y an idiomaticJSONnamelike total_count foraGoﬁeldnamed TotalCount.The tag

for Color has an addition alopt ion, omitempty,whichindic ates thatnoJSONout put should

be pro duce d if the ﬁeldhas the zerovalue for itstyp e (false,here) orisother wis e empt y.

Sure enoug h, theJSONout put for Casabl anca,ablack-and-w hite mov ie, has no color ﬁeld.

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

110 CHAPTER 4. COMPOSITE TYPES

Theinv ers e op erat iontomarsh aling, decodingJSONand popu lat ingaGodat a st ruc ture, is

called unmarshaling,and itisdon e by json.Unmarshal.The codebelow unmarsh als the

JSONmov iedat a into a slice ofstr uctswhose onlyﬁeldis Title.BydeﬁningsuitableGodat a

st ruc tures in thisway,wecan selec t whichpar ts of the JSONinp uttodecodeand whichtodis-

card . Wh en Unmarshal returns, ithas ﬁlled inthe slice wit h the Title infor mat ion; other

names in the JSONare ignored.

var titles []struct{ Title string }

if err := json.Unmarshal(data, &titles); err != nil {

log.Fatalf("JSON unmarshaling failed: %s", err)

}

fmt.Println(titles) // "[{Casablanca} {Cool Hand Luke} {Bullitt}]"

Many web ser vices provide a JSONint erface—make a requestwit h HT TP andbackcom esthe

desired infor mat ioninJSONfor mat. Toillustrate, let’squer y theGitHu b issuetrackerusing

itsweb-s ervice interface.First we’ll deﬁne the necessary typ es andcon stants:

gopl.io/ch4/github

// Package github provides a Go API for the GitHub issue tracker.

// See https://developer.github.com/v3/search/#search-issues.

package github

import "time"

const IssuesURL = "https://api.github.com/search/issues"

type IssuesSearchResult struct {

TotalCount int `json:"total_count"`

Items []*Issue

}

type Issue struct {

Number int

HTMLURL string `json:"html_url"`

Title string

State string

User *User

CreatedAt time.Time `json:"created_at"`

Body string // in Markdown format

}

type User struct {

HTMLURL string `json:"html_url"`

}

As before, the names ofall the str uct ﬁelds mustbecapit alize d even if their JSONnames are

not. How ever, the matchingpro cessthatass oci ates JSONnames wit h Go str uct names dur ing

unmarsh alingiscas e-insensit ive , so it’sonlynecessary touse a ﬁeldtag whenthere’sanunder-

scoreinthe JSONnamebut not inthe Goname. Again, weare beingselec tive about which

ﬁelds todecode; the GitHu b search respons e cont ainscon siderably moreinfor mat ionthanwe

showhere.

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

SECTION 4.5. JSON 111

The SearchIssues func tionmakes an HTTP requestand decodes the resultasJSON. Since

thequer y termspresent edbyausercou ldcontain charac terslike ? and & that havespeci al

me aning in a URL, weuse url.QueryEscape to ens ure thatthe y aretaken literal ly.

gopl.io/ch4/github

package github

import (

"encoding/json"

"fmt"

"net/http"

"net/url"

"strings"

)

// SearchIssues queries the GitHub issue tracker.

func SearchIssues(terms []string) (*IssuesSearchResult, error) {

q:=url.QueryEscape(strings.Join(terms, " "))

resp, err := http.Get(IssuesURL + "?q=" + q)

if err != nil {

return nil, err

}

// We must close resp.Body on all execution paths.

// (Chapter 5 presents 'defer',which makes this simpler.)

if resp.StatusCode != http.StatusOK {

resp.Body.Close()

return nil, fmt.Errorf("search query failed: %s", resp.Status)

}

var result IssuesSearchResult

if err := json.NewDecoder(resp.Body).Decode(&result); err != nil {

resp.Body.Close()

return nil, err

}

resp.Body.Close()

return &result, nil

}

Theearlier examples used json.Unmarshal to decodethe ent ire contentsofabyteslice as a

singleJSONent ity.For var iety, thisexampleusesthe st reami n g de coder, json.Decoder,

whichallowsseveral JSONent ities tobedecoded insequence fro m thesamestream, although

we don’tneed thatfeature here. Asyou mig htexp ect,there isacor respondingstreaming

enco der called json.Encoder.

Thecal l to Decode popu lates the var iable result.There are var ious ways wecan for mat its

value nicely.The simplest, demon strated bythe issues commandbelow,isasatext table

with ﬁxe d-w idth columns, but inthe next sec tion we’ll see a moresop histicate d approach

basedontempl ates.

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

112 CHAPTER 4. COMPOSITE TYPES

gopl.io/ch4/issues

// Issues prints a table of GitHub issues matching the search terms.

package main

import (

"fmt"

"log"

"os"

"gopl.io/ch4/github"

)

func main() {

result, err := github.SearchIssues(os.Args[1:])

if err != nil {

log.Fatal(err)

}

fmt.Printf("%d issues:\n", result.TotalCount)

for _, item := range result.Items {

fmt.Printf("#%-5d %9.9s %.55s\n",

item.Number, item.User.Login, item.Title)

}

Thecommand-linearguments specif y thesearc h terms. Thecommand below quer ies theGo

proj e ct’sissuetrackerfor the listofopenbugs rel ate d to JSONdecoding:

$gobuild gopl.io/ch4/issues

$./issues repo:golang/go is:open json decoder

13 issues:

#5680 eaigner encoding/json: set key converter on en/decoder

#6050 gopherbot encoding/json: provide tokenizer

#8658 gopherbot encoding/json: use bufio

#8462 kortschak encoding/json: UnmarshalText confuses json.Unmarshal

#5901 rsc encoding/json: allow override type marshaling

#9812 klauspost encoding/json: string tag not symmetric

#7872 extempora encoding/json: Encoder internally buffers full output

#9650 cespare encoding/json: Decoding gives errPhase when unmarshalin

#6716 gopherbot encoding/json: include field name in unmarshal error me

#6901 lukescott encoding/json, encoding/xml: option to treat unknown fi

#6384 joeshaw encoding/json: encode precise floating point integers u

#6647 btracey x/tools/cmd/godoc: display type kind of each named type

#4237 gjemiller encoding/base64: URLEncoding padding is optional

TheGitHu b we b-s ervice interface at https://developer.github.com/v3/ hasmanymore

fe aturesthanwehavespace for here.

Exercis e 4.10: Mo dif y issues to rep ort the results in agecategor ies, say lessthanamonth old,

lessthanayearold,and more thanayearold.

Exercis e 4.11: Buildatoolthatlets users cre ate, read, update, and deleteGitHu b issues fro m

thecommand line, inv oking their preferred text editorwhensubst ant ial text inputisrequired.

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

SECTION 4.6. TEXT AND HTML TEMPLATES 113

Exercis e 4.12: Thepopu lar web comic xkcd hasaJSONint erface.For example, a requestto

https://xkcd.com/571/info.0.json produces a detai le d des crip tionofcomic 571, one of

many favor ites. Dow nlo ad each URL (once!) andbui ld an ofﬂineindex. Write a tool xkcd

that, usingthisindex, printsthe URL andtranscript ofeachcomic thatmatch esasearc h term

prov ide d on the command line.

Exercis e 4.13: TheJSON-b ased web ser vice ofthe OpenMov ieDat abas e lets you searc h

https://omdbapi.com/ foramov iebynameand dow nlo ad itsposterimage . Wr ite a tool

poster that down loads the posterimage for the movienamed onthe command line.

4.6. Textand HTML Templates

Thepre vious exampledoesonlythe simplestpossiblefor matting , forwhich Printf is ent ire ly

ade quate. But som etimesfor matting mustbemoreelaborate, and it’sdesirable tosep aratethe

format fro m thecodemorecompletely. Thiscan bedon e with the text/template and

html/template packages, whichprovide a mechanism for subst ituting the values ofvar iables

into a text orHTMLtempl ate.

Atempl ate isastr ing orﬁle cont ainingone ormorepor tions enclos edindou ble braces,

{{...}},cal le d ac tions.Mostofthe str ing isprint edlit eral ly, but the actions trigger other

behavior s.Eachactioncontainsanexpressioninthe templ ate langu age, a simplebut pow erful

notation for printing values, selec ting str uct ﬁelds,cal lingfunctions and methods, expressing

cont rol ﬂow suchas if-else st atementsand range lo ops, andins tantiat ingother templ ates.

Asimpletempl ate str ing isshown below :

gopl.io/ch4/issuesreport

const templ = `{{.TotalCount}} issues:

{{range .Items}}----------------------------------------

Number: {{.Number}}

User: {{.User.Login}}

Title: {{.Title |printf "%.64s"}}

Age: {{.CreatedAt |daysAgo}} days

{{end}}`

Thistempl ate ﬁrs t pr intsthe numberofmatchingissues, thenprintsthe number, user, tit le,

andage indays ofeachone.Wit hin an action,there isanot ion of the cur rentvalue,refer red

to as ‘‘dot’’ andwritt enas ‘‘.’’,aper iod. The dot initial lyreferstothe templ ate’sparameter,

whichwill beagithub.IssuesSearchResult in thisexample. The {{.TotalCount}} ac tion

exp ands tothe value ofthe TotalCount ﬁeld, print edinthe usu alway.The

{{range .Items}} and {{end}} ac tions cre ate a loop, sothe text bet weenthemisexp ande d

mu ltipletimes, wit h dotbound tosuccessive elements of Items.

Wi thin an action,the | notation makes the resultofone operat ionthe argumentofanother,

analogou s to a Unix shel l pip eline. Inthe cas e of Title,the secon d op erat ionisthe printf

func tion, whichisabui lt-in synony m for fmt.Sprintf in alltempl ates. For Age,the secon d

op erat ionisthe fol low ing function, daysAgo,whichconvertsthe CreatedAt ﬁeldint o an

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

114 CHAPTER 4. COMPOSITE TYPES

el aps edtime, using time.Since:

func daysAgo(t time.Time) int {

return int(time.Since(t).Hours() / 24)

}

No tice thatthe typ e of CreatedAt is time.Time,not string.Inthe sameway thatatyp e may

cont rol its str ing for matting (§2.5) bydeﬁningcer tain met hods, a typ e mayals o deﬁne meth-

odstocontrol its JSONmarsh alingand unmarsh alingbeh avior.The JSON-marsh ale d value

of a time.Time is a str ing inastand ard for mat.

Producingout put wit h atempl ate isatwo-steppro cess. First wemustparse the templ ate int o

asuitableint ernal represent ation,and thenexe cut e it on speciﬁcinp uts. Parsingneed bedon e

on lyonce. The codebelow cre atesand parsesthe templ ate templ deﬁne d ab ove . No tethe

ch ainingofmet hod cal ls: template.New createsand retur nsatempl ate; Funcs adds daysAgo

to the set offunctions accessiblewit hin thistempl ate, thenretur nsthattempl ate;ﬁnally, Parse

is cal le d on the result.

report, err := template.New("report").

Funcs(template.FuncMap{"daysAgo": daysAgo}).

Parse(templ)

if err != nil {

log.Fatal(err)

}

Becaus e temp lates areusu allyﬁxe d at comp ile time, fai luretoparse a templ ate indic ates a fatal

buginthe program. The template.Must helperfunctionmakes erro r hand lingmorecon-

venient: itaccepts a templ ate and anerror,che cks thatthe error isnil (andpanics other wis e),

andthenretur nsthe templ ate. We’ll com e back to thiside a in Sec tion 5.9.

Once the templ ate has beencre ate d, aug mente d with daysAgo,parsed, and che cke d,wecan

exec ute itusingagithub.IssuesSearchResult as the dat a source and os.Stdout as the des-

tination:

var report = template.Must(template.New("issuelist").

Funcs(template.FuncMap{"daysAgo": daysAgo}).

Parse(templ))

func main() {

result, err := github.SearchIssues(os.Args[1:])

if err != nil {

log.Fatal(err)

}

if err := report.Execute(os.Stdout, result); err != nil {

log.Fatal(err)

}

Theprogram printsaplain text rep ort likethis:

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

SECTION 4.6. TEXT AND HTML TEMPLATES 115

$gobuild gopl.io/ch4/issuesreport

$./issuesreport repo:golang/go is:open json decoder

13 issues:

----------------------------------------

Number: 5680

User: eaigner

Title: encoding/json: set key converter on en/decoder

Age: 750 days

----------------------------------------

Number: 6050

User: gopherbot

Title: encoding/json: provide tokenizer

Age: 695 days

----------------------------------------

...

No w let’stur n to the html/template package. Itusesthe sameAPI and expressionlangu age

as text/template butaddsfeaturesfor aut omat ic andcontext-appropriateescapingofstr ings

appear ingwit hin HTML, JavaS crip t,CSS, orURLs. Thes e fe aturescan helpavoid a perenni al

security pro blem ofHTMLgenerat ion, an injectionattack, in which an adversary craf ts a

st ringvalue like the tit leofanissuetoinclude malicious codethat, whenimp rop erlyescap ed

by a templ ate, gives themcontrol overthe page.

Thetempl ate below printsthe listofissues as an HTML table.Not e thedif ferentimp ort:

gopl.io/ch4/issueshtml

import "html/template"

var issueList = template.Must(template.New("issuelist").Parse(`

<h1>{{.TotalCount}} issues</h1>

<table>

<th>State</th>

<th>Title</th>

</tr>

<tr>

<td><a href='{{.HTMLURL}}'>{{.Number}}</td>

<td>{{.State}}</td>

<td><a href='{{.User.HTMLURL}}'>{{.User.Login}}</a></td>

<td><a href='{{.HTMLURL}}'>{{.Title}}</a></td>

</tr>

</table>

`))

Thecommand below exe cut esthe newtempl ate onthe results ofaslig htlydif ferentquer y:

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

116 CHAPTER 4. COMPOSITE TYPES

$gobuild gopl.io/ch4/issueshtml

$./issueshtml repo:golang/go commenter:gopherbot json encoder >issues.html

Figure4.4 shows the app earance ofthe table in a web brows er.The lin ks connec t to the

appropriateweb pages at GitHu b.

Figure 4.4. An HTMLtable ofGopro jec t issues rel ating toJSONencoding.

No neofthe issues in Figure4.4 pos e achallenge for HTML, but wecan see the effec t more

clearlywit h issues whose tit les cont ain HTML met acharac terslike & and <.We’ve selec ted two

such issues for thisexample:

$./issueshtml repo:golang/go 3133 10535 >issues2.html

Figure4.5 shows the resultofthisquer y.Not ice thatthe html/template packageaut omat i-

callyHTML-es cap edthe tit les sothatthe y appear literal ly. Had weusedthe text/template

packagebymistake , thefour-ch arac ter str ing "<" wouldhavebeenrendered asaless-t han

ch arac ter '<',and the str ing "<link>" wouldhavebecom e a link element, chang ing the

st ruc tureofthe HTMLdocumentand perhaps comp romisingits sec urity.

We can sup pressthisaut o-es capingbeh avior for ﬁelds thatcontain trusted HTMLdat a by

usingthe named str ing typ e template.HTML insteadof string.Simi lar named typ es exist for

tr usted JavaS crip t,CSS, andURLs. Theprogram below demon strates the princip lebyusing

twoﬁelds wit h thesamevalue but dif ferenttyp es: A is a str ing and B is a template.HTML.

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

SECTION 4.6. TEXT AND HTML TEMPLATES 117

Figure 4.5. HTML met acharac ters in issuetit les arecor rec tly displ aye d.

gopl.io/ch4/autoescape

func main() {

const templ = `<p>A: {{.A}}</p><p>B: {{.B}}</p>`

t:=template.Must(template.New("escape").Parse(templ))

var data struct {

Astring // untrusted plain text

Btemplate.HTML // trusted HTML

}

data.A = "<b>Hello!</b>"

data.B = "<b>Hello!</b>"

if err := t.Execute(os.Stdout, data); err != nil {

log.Fatal(err)

}

Figure4.6 shows the templ ate’sout put asitapp earsinabrows er.Wecan see that A was

su bjec t to escapingbut B was not.

Figure 4.6. St ringvalues areHTML-es cap edbut template.HTML values arenot.

We havespace heretoshowonlythe most basic featuresofthe templ ate system. Asalways, for

more infor mat ion, cons ult the packagedocumentation:

$godoc text/template

$godoc html/template

Exercis e 4.14: Create a web ser ver thatquer ies GitHub onceand thenallowsnav igationofthe

list ofbug rep orts, milestones, andusers.

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

This page intentionally left blank

From the Library of YIGUANG HU

ptg16105617

Functions

Afunctionlets uswrap upasequence ofstatementsasaunitthatcan becal le d from els e where

in a program, perhaps multipletimes. Functions makeitpossibletobre akabig job int o

smal ler pieces thatmig htwel l be writt enbydif ferentpeoplesep arated bybot h time and space.

Afunctionhides its implementation det ails fro m itsusers.For all ofthese reasons,functions

areacriticalpar t of any programminglangu age.

We’ve seenmanyfunctions already.Now let’staketimefor a morethoro ugh dis cussion. The

runningexampleofthischapt erisaweb craw ler,thatis, the component ofaweb searc h

engine responsible for fetchingweb pages, discov ering the lin ks within them, fetchingthe

pagesidentiﬁed bythose lin ks, andsoon. A we b craw ler givesusamp leopp ortunity to

explore recursion, anonymou s func tions,error handling, and asp ectsoffunctions thatare

unique toGo.

5.1. Function Declarations

Afunctiondeclarat ionhas a name, a listofparameters, an opt ion allistofresults, andabody:

func

name

(

parameter-list

)(

result-list

){

body

}

Theparameter list speciﬁes the names andtyp es of the function’s para meters,whichare the

lo cal variables whose values or ar gum ents aresup plie d by the cal ler.The resultlistspeciﬁes

thetyp es of the values thatthe functionretur ns. Ifthe functionretur nsone unnamed result

or noresults at all, parenthes es areopt ion aland usu allyomitt ed. Leaving off the resultlist

entire lydeclaresafunctionthatdoesnot retur n anyvalue andiscal le d on lyfor itseffec ts. In

the hypot func tion,

119

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

120 CHAPTER 5. FUNCTIONS

func hypot(x, y float64) float64 {

return math.Sqrt(x*x + y*y)

}

fmt.Println(hypot(3, 4)) // "5"

x and y areparametersinthe declarat ion, 3 and 4 arearguments ofthe cal l,and the function

returnsafloat64 value.

Like parameters, results may benamed.Inthatcas e,eachnamedeclaresalocal variableini-

tialize d to the zerovalue for itstyp e.

Afunctionthathas a resultlistmustend wit h a return st atement unlessexe cut ion cle arly

cannot reach the end ofthe function, perhaps because the functionend s with a cal l to panic

or aninﬁnite for lo opwit h no break.

As wesaw wit h hypot,asequence ofparametersorresults ofthe sametyp e canbefac tored so

that the typ e itself iswritt enonlyonce. These two declarat ions are equivalent:

func f(i, j, k int, s, t string) {/*... */ }

func f(i int, j int, k int, s string, t string) { /* ... */ }

Here are fourways todeclare a functionwit h twoparametersand one result, alloftyp e int.

Theblank identiﬁer can beusedtoemp hasize thataparameter isunu sed.

func add(x int, y int) int {return x + y }

func sub(x, y int) (z int) {z=x-y;return }

func first(x int, _ int) int { return x }

func zero(int, int) int {return 0 }

fmt.Printf("%T\n", add) // "func(int, int) int"

fmt.Printf("%T\n", sub) // "func(int, int) int"

fmt.Printf("%T\n", first) // "func(int, int) int"

fmt.Printf("%T\n", zero) // "func(int, int) int"

Thetyp e of a functionissom etimescal le d its sig nat ure.Two functions havethe sametyp e or

sig nature if the y have the samesequence ofparameter typ es andthe samesequence ofresult

types. Thenames ofparametersand results don’taffec t thetyp e,nor doeswhether ornot the y

were declare d usingthe fac tored for m.

Ev ery functioncal l mu stprovide an argumentfor eachparameter,inthe order in whichthe

parametersweredeclare d.Gohas nocon ceptofdefau ltparameter values, nor any way to

sp ecif y arguments byname, sothe names ofparametersand results don’tmattertothe cal ler

except as documentation.

Parametersare local variables wit hin the bodyofthe function, wit h their initial values set to

thearguments sup plie d by the cal ler.Functionparametersand named results arevar iables in

thesamelexic al block as the function’sout ermostlocal variables.

Arguments arepassed by value,sothe functionreceivesacopyofeachargument; modiﬁca-

tion s to the copy do not affec t thecal ler.How ever, ifthe argumentcontainssom e kind ofref-

erence,likea point er, slice,map,function, orchannel,thenthe cal ler maybeaffec ted byany

mo diﬁcation s thefunctionmakes tovar iables in direc tly referred tobythe argument.

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

SECTION 5.2. RECURSION 121

Yo u mayocc asionallyencount erafunctiondeclarat ionwit houtabody, indic atingthatthe

func tionisimp lemente d in a langu ageother thanGo. Suchadeclarat iondeﬁnesthe function

sig nature.

package math

func Sin(x float64) float64 // implemented in assembly language

5.2. Recursion

Func tions may be re cur siv e,thatis, the y maycal l themselves, either direc tly orindirec tly.

Re cursionisapow erfultechnique for manypro blems, andofcours e it’sess ent ial for process-

ingrec ursivedat a st ruc tures. InSec tion 4.4, weusedrec ursionoveratree toimp lementa

simpleins ertionsor t.Inthissec tion,we’ll use itagain for processingHTMLdocuments.

Theexampleprogram below usesanon-stand ard package, golang.org/x/net/html,which

prov ides an HTML parser. The golang.org/x/... repositories holdpackages designe d and

maintained bythe Goteamfor app lic ations suchasnet wor king, int ernat ionalize d text

processing, mobileplatfor ms, imagemanipu lat ion, cryptography,and develop ertools.These

packages arenot inthe stand ard librar y becaus e they’re still under deve lopment orbecause

they’re rarelyneeded bythe maj ority ofGoprogrammers.

Thepar ts of the golang.org/x/net/html APIthatwe’ll need are shown below.The function

html.Parse re ads a sequence ofbytes, parsesthem, andretur nsthe rootofthe HTMLdoc-

umenttre e,whichisan html.Node.HTMLhas several kinds ofnodes—text, comments, and

so on—buthereweare con cer ned onlywit h el ement no des ofthe for m <name key='value'>.

golang.org/x/net/html

package html

type Node struct {

Type NodeType

Data string

Attr []Attribute

FirstChild, NextSibling *Node

}

type NodeType int32

const (

ErrorNode NodeType = iota

TextNode

DocumentNode

ElementNode

CommentNode

DoctypeNode

)

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

122 CHAPTER 5. FUNCTIONS

type Attribute struct {

Key, Val string

}

func Parse(r io.Reader) (*Node, error)

The main func tionparsesthe stand ard inp utasHTML, extrac ts thelin ks usingarec ursive

visit func tion, andprintseachdis cov ere d lin k:

gopl.io/ch5/findlinks1

// Findlinks1 prints the links in an HTML document read from standard input.

package main

import (

"fmt"

"os"

"golang.org/x/net/html"

)

func main() {

doc, err := html.Parse(os.Stdin)

if err != nil {

fmt.Fprintf(os.Stderr, "findlinks1: %v\n", err)

os.Exit(1)

}

for _, link := range visit(nil, doc) {

fmt.Println(link)

}

The visit func tiontraversesanHTMLnodetre e,ext racts the lin k from the href attr ibute of

each an chor element <a href='...'>,app end s thelin ks to a slice ofstr ings, andretur nsthe

resu lting slice:

// visit appends to links each link found in n and returns the result.

func visit(links []string, n *html.Node) []string {

if n.Type == html.ElementNode && n.Data == "a" {

for _, a := range n.Attr {

if a.Key == "href" {

links = append(links, a.Val)

}

for c := n.FirstChild; c != nil; c = c.NextSibling {

links = visit(links, c)

}

return links

}

To des cendthe tre e fora node n, visit re cursive lycal lsits elf for eachof n’schi ldren, which

areheld in the FirstChild lin ked list.

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

SECTION 5.2. RECURSION 123

Let’srun findlinks on the Gohom e page , piping the out put of fetch (§1.5) tothe inp utof

findlinks.We’ve edite d theout put slig htlyfor brevit y.

$gobuild gopl.io/ch1/fetch

$gobuild gopl.io/ch5/findlinks1

$./fetch https://golang.org | ./findlinks1

/doc/

/pkg/

/help/

/blog/

http://play.golang.org/

//tour.golang.org/

https://golang.org/dl/

//blog.golang.org/

/LICENSE

/doc/tos.html

http://www.google.com/intl/en/policies/privacy/

No tice the var ietyoffor msoflin ks that appear in the page. Later we’ll see how tores olve

them rel ative tothe bas e URL, https://golang.org,tomakeabs olut e URLs.

Thenext program usesrec ursionoverthe HTMLnodetre e to print the str uctureofthe tre e in

outline. Asitencount ers eachelement, itpushesthe element’stag ont o astack,thenprintsthe

st ack.

gopl.io/ch5/outline

func main() {

doc, err := html.Parse(os.Stdin)

if err != nil {

fmt.Fprintf(os.Stderr, "outline: %v\n", err)

os.Exit(1)

}

outline(nil, doc)

}

func outline(stack []string, n *html.Node) {

if n.Type == html.ElementNode {

stack = append(stack, n.Data) // push tag

fmt.Println(stack)

}

for c := n.FirstChild; c != nil; c = c.NextSibling {

outline(stack, c)

}

No teone subtlet y:alt hough outline ‘‘pu shes’’ an elementon stack,there isnocor responding

pop. When outline callsits elf rec ursively, the cal le e re ceivesacopyof stack.Alt hough the

callee may append elements tothisslice,modif yingits underly ing array andperhaps even

al locating a new array,itdoesn’tmodif y theinitial elements thatare visible tothe cal ler,so

when the functionretur ns, the cal ler’s stack is asitwas beforethe cal l.

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

124 CHAPTER 5. FUNCTIONS

Here’sthe out lineof https://golang.org,again edite d forbre vit y:

$gobuild gopl.io/ch5/outline

$./fetch https://golang.org | ./outline

[html]

[html head]

[html head meta]

[html head title]

[html head link]

[html body]

[html body div]

[html body div div]

[html body div div form]

[html body div div form div]

[html body div div form div a]

...

As you can see byexp erimenting wit h outline,mostHTMLdocuments can bepro cessed

with onlyafew level s of rec ursion, but it’snot hardtocon str uct pat holog ical web pages that

re quireext remelydeeprec ursion.

Many programminglangu ageimp lementation s us e aﬁxe d-size functioncal l st ack;sizes fro m

64KB to2MB aretypic al.Fixed-size stacksimp ose a limitonthe depth ofrec ursion, soone

mu stbecaref ultoavoid a sta ckoverﬂow when traversinglarge dat a st ruc tures rec ursively;

ﬁxe d-size stacksmay evenpos e asec urity risk. Incontrast, typic al Go imp lementation s us e

var iable-size stacksthatstart small and grow as needed uptoalimitonthe order ofagigabyte.

Thislets ususe rec ursionsafelyand wit houtwor rying about overﬂow.

Exercis e 5.1: Change the findlinks prog ram to traverse the n.FirstChild lin ked listusing

re cursive cal lsto visit insteadofaloop.

Exercis e 5.2: Wr ite a functiontopopu lateamapping fro m elementnames—p, div, span,and

so on—to the numberofelements wit h that name in an HTMLdocumenttre e.

Exercis e 5.3: Wr ite a functiontoprint the contentsofall text nodes in an HTML document

tree.Donot des cendint o <script> or <style> elements, since their cont entsare not visible

in a web brows er.

Exercis e 5.4: Extend the visit func tionsothatitext racts other kinds oflin ks from the doc-

ument, such as images, scr ipts, andsty leshe ets.

5.3. Multiple Return Values

Afunctioncan retur n more thanone result. We’ve seenmanyexamples offunctions fro m

st and ard packages thatretur n twovalues, the desired computation alresultand anerror value

or boole an that indic ates whether the computation wor ked.The next exampleshows how to

wr ite one ofour own.

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

SECTION 5.3. MULTIPLE RETURN VALUES 125

Theprogram below isavar iat ionof findlinks that makes the HTTPrequestits elf sothatwe

no lon g erneed torun fetch.Because the HTTPand parsingoperat ions can fail, findLinks

de clarestwo results: the listofdis cov ere d lin ks andanerror.Incidentally, the HTMLparser

canusu allyrecov erfro m badinp utand con str uct a documentcontainingerror nodes, so

Parse rarelyfai ls; whenitdoes, it’stypic ally due tounderly ing I/O erro rs.

gopl.io/ch5/findlinks2

func main() {

for _, url := range os.Args[1:] {

links, err := findLinks(url)

if err != nil {

fmt.Fprintf(os.Stderr, "findlinks2: %v\n", err)

continue

}

for _, link := range links {

fmt.Println(link)

}

// findLinks performs an HTTP GET request for url, parses the

// response as HTML, and extracts and returns the links.

func findLinks(url string) ([]string, error) {

resp, err := http.Get(url)

if err != nil {

return nil, err

}

if resp.StatusCode != http.StatusOK {

resp.Body.Close()

return nil, fmt.Errorf("getting %s: %s", url, resp.Status)

}

doc, err := html.Parse(resp.Body)

resp.Body.Close()

if err != nil {

return nil, fmt.Errorf("parsing %s as HTML: %v", url, err)

}

return visit(nil, doc), nil

}

Thereare fourretur n st atementsin findLinks,eachofwhichretur nsapair ofvalues. The

ﬁrs t thre e returnscause the functiontopassthe underly ing error s from the http and html

packages ontothe cal ler.Inthe ﬁrs t case,the error isretur ned unchange d;inthe secon d and

third, itisaug mente d with addition alcontext infor mat ionby fmt.Errorf (§7.8). If find-

Links is successf ul, the ﬁnalretur n st atement retur nsthe slice oflin ks, with noerror.

We mustens ure that resp.Body is clos edsothatnet wor k resources arepro perly releasedeven

in cas e of error.Go’sgarb agecol lec tor rec ycles unus edmemor y,but donot assume itwill

re lease unu sed operat ingsystemres ources like openﬁles andnet wor k connec tion s.The y

shouldbeclos edexplicitly.

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

126 CHAPTER 5. FUNCTIONS

Theresultofcal lingamulti-value d func tionisatup leofvalues. Thecal ler of suchafunction

mu stexplicitlyassig n thevalues tovar iables if anyofthemare tobeused:

links, err := findLinks(url)

To ignoreone ofthe values, assig n it tothe blank identiﬁer :

links, _ := findLinks(url) // errors ignored

Theresultofamulti-value d call may its elf beretur ned fro m a(mu lti-value d) callingfunction,

as in thisfunctionthatbeh aveslike findLinks butlogs its argument:

func findLinksLog(url string) ([]string, error) {

log.Printf("findLinks %s", url)

return findLinks(url)

}

Amulti-value d call may appear as the soleargumentwhencal lingafunctionofmultiple

parameters. Alt hough rarelyusedinpro duc tioncode, thisfeature issom etimesconvenient

during debug gingsince itlets usprint all the results ofacal l usingasinglestatement.The two

pr int statementsbelow havethe sameeffec t.

log.Println(findLinks(url))

links, err := findLinks(url)

log.Println(links, err)

We ll-chosennames can document the sig niﬁcanceofafunction’sresults. Names arepar tic u-

larlyvaluablewhenafunctionretur nsmultipleresults ofthe sametyp e,like

func Size(rect image.Rectangle) (width, height int)

func Split(path string) (dir, file string)

func HourMinSec(t time.Time) (hour, minute, second int)

butit’snot always necessary tonamemultipleresults solelyfor documentation.For ins tance,

conv ent ion dic tates thataﬁnal bool resu ltindic ates success; an error resu ltoften needsno

explanation.

In a functionwit h name d resu lts, the operands ofaretur n st atement may beomitt ed. Thisis

called a bare return.

// CountWordsAndImages does an HTTP GET request for the HTML

// document url and returns the number of words and images in it.

func CountWordsAndImages(url string) (words, images int, err error) {

resp, err := http.Get(url)

if err != nil {

return

}

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

SECTION 5.4. ERRORS 127

doc, err := html.Parse(resp.Body)

resp.Body.Close()

if err != nil {

err = fmt.Errorf("parsing HTML: %s", err)

return

}

words, images = countWordsAndImages(doc)

return

}

func countWordsAndImages(n *html.Node) (words, images int) { /* ... */ }

Abareretur n is a shorthand way toretur n each ofthe named resultvar iables in order,soin

thefunctionabove , each retur n st atement isequivalentto

return words, images, err

In functions likethisone,wit h many retur n st atementsand several results, bareretur nscan

re duce code dup lic ation, but the y rarelymakecodeeasier tounderst and.For ins tance,it’snot

obviou s at ﬁrs t glance thatthe two early retur nsare equivalentto return 0, 0, err (b ecaus e

theresultvar iables words and images areinitialize d to their zerovalues) andthatthe ﬁnal

return is equivalentto return words, images, nil.For thisreason, bareretur nsare best

us edsparing ly.

Exercis e 5.5: Implement countWordsAndImages.(SeeExercis e 4.9 for word-split ting.)

Exercis e 5.6: Mo dif y the corner func tionin gopl.io/ch3/surface (§3.2) touse named

resu lts andabareretur n st atement.

5.4. Errors

Some functions always succe e d at their task. For example, strings.Contains and str-

conv.FormatBool have well-deﬁne d resu lts for all possibleargumentvalues andcannot fai l—

barring cat astrop hic andunp redic table scenar ios li kerunningout of memor y,where the

sy mpt omisfar fro m thecause and fro m whichthere’slit tle hop e of recov ery.

Ot her functions always succe e d so lon g as their pre con dit ion s aremet. For example, the

time.Date func tionalways cons tructsatime.Time from itscomponents—ye ar, month,and

so on—un lessthe lastargument(thetimezon e)is nil,inwhichcas e it panics. Thispanic isa

sure sig n of a bug in the cal lingcodeand shouldnever happen in a wel l-w rit ten program.

Fo r many other functions,eveninawel l-w rit ten program, successisnot assure d becaus e it

dep ends onfac tor s beyond the programmer’scontrol . AnyfunctionthatdoesI/O,for exam-

ple, mustconfrontthe possibi lit y of error,and onlyanaïve programmer belie ves a simpleread

or write cannot fai l.Indeed,it’swhenthe most reliableoperat ions fai l unexp ectedlythatwe

most need toknowwhy.

Er ror s arethu s an importantpar t of a package’sAPI oranapp lic ation’suserint erface,and fai l-

ureisjustone ofseveral expec ted beh avior s.Thisisthe appro ach Gotakes toerror handling.

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

128 CHAPTER 5. FUNCTIONS

Afunctionfor whichfai lureisanexp ected beh avior retur nsanaddition alresult, conv ent ion-

al lythe lastone.Ifthe fai lurehas onlyone possiblecause,the resultisaboole an, usually

called ok,asinthisexampleofacache looku p that always succe e dsunlessthere was noent ry

forthatkey :

value, ok := cache.Lookup(key)

if !ok {

// ...cache[key] does not exist...

}

Mo reoften, andesp eci ally for I/O,the fai luremay haveavar ietyofcausesfor whichthe cal ler

wi l l ne e d an explanation.Insuchcas es, thetyp e of the addition alresultis error.

Thebui lt-in typ e error is anint erface typ e.We’ll see moreofwhatthismeans and its impli-

cation s forerror handlinginChapt er7.For now it’senoug h to knowthatan error maybenil

or non-ni l,thatnil imp lies successand non-ni l implies failure , andthatanon-ni l error has

an erro r mess age str ing whichwecan obt ain bycal lingits Error method orprint bycal ling

fmt.Println(err) or fmt.Printf("%v", err).

Us ual lywhenafunctionretur nsa non-ni l er ror,its other results areundeﬁne d andshouldbe

ig nored.How ever, a few functions may retur n partialresults in erro r cases. For example, ifan

er ror occ urswhi le re adingfro m a ﬁle,acal l to Read returnsthe numberofbytes itwas able to

re ad an d an error value descr ibing the pro blem. For cor rec t behavior,som e callersmay need

to pro cessthe incompletedat a before handlingthe error,soitisimp ortantthatsuchfunctions

clearlydocumenttheir results.

Go’sappro ach sets itapart from manyother langu ages in whichfai lures arerep orted using

ex cepti ons,not ordinar y values. Alt hough Godoeshaveanexception mechanism ofsor ts, as

we will see inSec tion 5.9, itisusedonlyfor rep ortingtruly unexp ected error s that indic atea

bug, not the routine error s that a robustprogram shouldbebui lttoexp ect.

Thereasonfor thisdesig n is thatexception s tend toent ang lethe des crip tionofanerror wit h

thecontrol ﬂow required tohandleit, often leadingtoanundesirable out com e:routine error s

arerep orted tothe end userinthe for m of anincomprehensible stack trace,full of

infor mat ionabout the str uctureofthe program but lacking int ellig iblecontext aboutwhat

went wro ng.

By contrast, Goprogramsuse ordinar y cont rol-ﬂow mechanismslike if and return to

resp ond toerror s.Thissty leundeni ably demands thatmoreatt ent ion bepaid toerror-han-

dlinglog ic, butthatispre cis ely the point.

5.4.1. Error-Handling Strategies

Wh enafunctioncal l returnsanerror,it’sthe cal ler’sresponsibilit y to checkitand take

appropriateaction. Dep endingonthe sit uat ion, there may beanumb erofpossibi lit ies. Let’s

take a lookatﬁve ofthem.

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

SECTION 5.4. ERRORS 129

First,and most common, isto prop agate theerror,sothatafai lureinasubro utine becom esa

fai lureofthe cal lingroutine.Wesaw examples ofthisinthe findLinks func tionof

Section5.3. Ifthe cal l to http.Get fai ls, findLinks returnsthe HTTPerror tothe cal ler

withoutfur therado:

resp, err := http.Get(url)

if err != nil {

return nil, err

}

In contrast, if the cal l to html.Parse fai ls, findLinks do es notretur n theHTMLparser’s

er ror direc tly because itlacks two cruci al pieces ofinfor mat ion: thatthe error occ urre d in the

pars er, and the URL ofthe documentthatwas beingparsed. Inthiscas e, findLinks con-

st ruc ts anew error message thatincludes bot h pieces ofinfor mat ionaswel l as the underly ing

pars e er ror :

doc, err := html.Parse(resp.Body)

resp.Body.Close()

if err != nil {

return nil, fmt.Errorf("parsing %s as HTML: %v", url, err)

}

The fmt.Errorf func tionfor mats an erro r mess age using fmt.Sprintf andretur nsanew

error value.Weuse ittobui ld des crip tiveerror s by successive lypreﬁxingaddition alcontext

infor mat iontothe originalerror message . Wh enthe error isult imate lyhandled bythe

prog ram’s main func tion, itshouldprovide a cle ar caus alchain fro m therootpro blem tothe

ov eral l fai lure, reminiscentofaNASAaccidentinv est igat ion:

genesis: crashed: no parachute: G-switch failed: bad relay orientation

Becaus e er ror messagesare fre quentlychained toget her,message str ingsshouldnot becapit al-

ize d andnewlines shouldbeavoide d.The resulting error s maybelon g,but the y wi l l be self-

cont ained whenfound bytools like grep.

Wh endesig ningerror messages, bedelib erate, sothateachone isameaningf ul des crip tionof

thepro blem wit h sufﬁcientand relevantdet ail, and becon sistent,sothaterror s returned by

thesamefunctionorbyagro upoffunctions inthe samepackageare simi lar in for m andcan

be dealt wit h in the sameway.

Fo r example, the os packageguarante esthatevery error retur ned byaﬁle operat ion, suchas

os.Open or the Read, Write,or Close methodsofanopen ﬁle,des crib esnot justthe natureof

thefai lure(permissiondenie d,nosuchdirec tor y,and soon) but als o thenameofthe ﬁle,so

thecal ler ne e dn’tinclude thisinfor mat ion in the error message itcon str ucts.

In general,the cal l f(x) is responsible for rep ortingthe att emp ted operat ion f andthe argu-

ment value x as the y re latetothe context ofthe error.The cal ler is responsible for addingfur-

ther infor mat ionthatithas but the cal l f(x) do es not, suchasthe URL in the cal l to

html.Parse ab ove .

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

130 CHAPTER 5. FUNCTIONS

Let’smov e on tothe secon d st rateg y forhandlingerror s.For error s that represent transientor

unpredic table pro blems, itmay makesense to re try thefai le d op erat ion, possibly wit h adel ay

betweentries, andperhaps wit h alimitonthe numberofatt emp tsorthe timespent trying

before giv ing upent ire ly.

gopl.io/ch5/wait

// WaitForServer attempts to contact the server of a URL.

// It tries for one minute using exponential back-off.

// It reports an error if all attempts fail.

func WaitForServer(url string) error {

const timeout = 1 * time.Minute

deadline := time.Now().Add(timeout)

for tries := 0; time.Now().Before(deadline); tries++ {

_, err := http.Head(url)

if err == nil {

return nil // success

}

log.Printf("server not responding (%s); retrying...", err)

time.Sleep(time.Second << uint(tries)) // exponential back-off

}

return fmt.Errorf("server %s failed to respond after %s", url, timeout)

}

Third, ifprogressisimp ossible,the cal ler canprint the error and stopthe program gracef ully,

butthiscours e of actionshouldgeneral lyberes erve d forthe main packageofaprogram.

Librar y func tions shouldusu allypro pagateerror s to the cal ler,unlessthe error isasig n of an

internal inconsistenc y—that is, a bug .

// (In function main.)

if err := WaitForServer(url); err != nil {

fmt.Fprintf(os.Stderr, "Site is down: %v\n", err)

os.Exit(1)

}

Amoreconvenientway toachie vethe sameeffec t is tocal l log.Fatalf.Aswit h al l the log

func tions,bydefau ltitpreﬁxesthe timeand datetothe error message .

if err := WaitForServer(url); err != nil {

log.Fatalf("Site is down: %v\n", err)

}

Thedefau ltfor mat ishelpf ul in a lon g-r unningser ver,but lesssofor anint erac tivetool:

2006/01/02 15:04:05 Site is down: no such domain: bad.gopl.io

Fo r amoreatt rac tiveout put,wecan set the preﬁxusedbythe log packagetothe nameofthe

command, and sup pressthe displ ayofthe dateand time:

log.SetPrefix("wait: ")

log.SetFlags(0)

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

SECTION 5.4. ERRORS 131

Fo urth, in som e cases, it’ssufﬁcientjusttolog the error and thencontinue, perhaps wit h

re duce d func tionality.Again there’sachoice bet weenusingthe log package, whichaddsthe

usualpreﬁx:

if err := Ping(); err != nil {

log.Printf("ping failed: %v; networking disabled", err)

}

andprint ing direc tly tothe stand ard error stream:

if err := Ping(); err != nil {

fmt.Fprintf(os.Stderr, "ping failed: %v; networking disabled\n", err)

}

(A ll log func tions app end a newline if one isnot already present.)

Andﬁfth and ﬁnally, inrarecas es we can safelyignore an error ent ire ly:

dir, err := ioutil.TempDir("", "scratch")

if err != nil {

return fmt.Errorf("failed to create temp dir: %v", err)

}

// ...use temp dir...

os.RemoveAll(dir) // ignore errors; $TMPDIR is cleaned periodically

Thecal l to os.RemoveAll mayfai l,but the program ignoresitbecause the operat ingsystem

periodic ally cle ansout the temporar y direc tor y.Inthiscas e,dis cardingthe error was inten-

tion al, but the program logic wou ldbethe samehad wefor g ott entodealwit h it.Get into the

habit of con sider ingerror s af ter every functioncal l,and whenyou delib eratelyignoreone,

do cumentyourint ent ion cle arly.

Er ror handlinginGohas a par tic ularrhythm. After che cking anerror,fai lureisusu allydealt

with beforesuccess. Iffai lurecausesthe functiontoretur n, thelog ic forsuccessisnot

indente d within an else blockbut fol lowsatthe out erlevel.Functions tendtoexhibit a com-

monstr ucture, wit h aser ies of initial che cks torej e cterror s,fol low edbythe subst anceofthe

func tion at the end,minimal lyindente d.

5.4.2. End ofFile (EOF)

Us ual ly, the var ietyoferror s that a functionmay retur n is int erest ing tothe end userbut not

to the int erveningprogram logic. On occasion, how ever, a program musttakedif ferent

ac tions dep endingonthe kindoferror thathas occ urre d.Con sider an attemp t to read n bytes

of dat a from a ﬁle.If n is chosentobethe lengt h of the ﬁle,any error representsafai lure. On

theother hand, ifthe cal ler repeatedlytries toreadﬁxe d-size chunksunt i l theﬁle isexhausted,

thecal ler mu strespond dif ferentlytoanend-of-ﬁle condit ion thanitdoestoall other erro rs.

Fo r this reason, the io packageguarante esthatany readfai lurecausedbyanend-of-ﬁle condi-

tion isalways rep orted byadistinguishe d er ror, io.EOF,whichisdeﬁne d as fol lows:

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

132 CHAPTER 5. FUNCTIONS

package io

import "errors"

// EOF is the error returned by Read when no more input is available.

var EOF = errors.New("EOF")

Thecal ler candetec t this con dit ion usingasimplecomparison, as in the loopbelow,which

re ads runes fro m thestand ard inp ut. (The charcount prog ram in Sec tion 4.3 provides a more

comp leteexample.)

in := bufio.NewReader(os.Stdin)

for {

r, _, err := in.ReadRune()

if err == io.EOF {

break // finished reading

}

if err != nil {

return fmt.Errorf("read failed: %v", err)

}

// ...use r...

}

Sinceinanend-of-ﬁle condit ion there isnoinfor mat iontorep ort besides the fac t of it, io.EOF

hasaﬁxe d er ror message , "EOF".For other erro rs, wemay need torep ort bot h thequality and

qu ant ity ofthe error,sotospeak, soaﬁxe d er ror value will not do. InSec tion 7.11, we’ll

pres entamoresystematicway todistinguish cer tain erro r values fro m ot hers.

5.5. Function Values

Func tions are ﬁrst-class values in Go: like other values, functionvalues havetyp es, andthe y

maybeassig ned tovar iables orpassedtoorretur ned fro m func tions.Afunc tionvalue may

be cal le d li keany other function. For example:

func square(n int) int {return n * n }

func negative(n int) int {return -n }

func product(m, n int) int { return m * n }

f:=square

fmt.Println(f(3)) // "9"

f=negative

fmt.Println(f(3)) // "-3"

fmt.Printf("%T\n", f) // "func(int) int"

f=product // compile error: can'tassign f(int, int) int to f(int) int

Thezerovalue ofafunctiontyp e is nil.Cal linganil functionvalue causesapanic:

var f func(int) int

f(3) // panic: call of nil function

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

SECTION 5.5. FUNCTION VALUES 133

Func tionvalues may becompare d with nil:

var f func(int) int

if f != nil {

f(3)

}

butthe y arenot comparable,sothe y maynot becompare d against eachother orusedaskeys

in a map.

Func tionvalues let usparameter ize ourfunctions overnot justdat a, butbeh avior too.The

st and ard librar ies cont ain manyexamples. For ins tance, strings.Map applies a functionto

each charac ter ofastr ing , joiningthe results tomakeanother str ing .

func add1(r rune) rune { return r + 1 }

fmt.Println(strings.Map(add1, "HAL-9000")) // "IBM.:111"

fmt.Println(strings.Map(add1, "VMS")) // "WNT"

fmt.Println(strings.Map(add1, "Admix")) // "Benjy"

The findLinks func tionfro m Section5.2 usesahelperfunction, visit,tovisit all the nodes

in an HTML document and app l y an action toeachone.Usingafunctionvalue,wecan sep a-

ratethe log ic fortre e traversalfro m thelog ic forthe actiontobeapp lie d to eachnode, letting

us reuse the traversalwit h dif ferentactions.

gopl.io/ch5/outline2

// forEachNode calls the functions pre(x) and post(x) for each node

// x in the tree rooted at n. Both functions are optional.

// pre is called before the children are visited (preorder) and

// post is called after (postorder).

func forEachNode(n *html.Node, pre, post func(n *html.Node)) {

if pre != nil {

pre(n)

}

for c := n.FirstChild; c != nil; c = c.NextSibling {

forEachNode(c, pre, post)

}

if post != nil {

post(n)

}

The forEachNode func tionaccepts two functionarguments, one tocal l before a node’schi l-

dren arevisit edand one tocal l af ter.Thisarrangement gives the cal ler agre atdealof ﬂexi-

bi lit y.For example, the functions startElement and endElement pr int the start and end tags

of anHTMLelementlike <b>...</b>:

var depth int

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

134 CHAPTER 5. FUNCTIONS

func startElement(n *html.Node) {

if n.Type == html.ElementNode {

fmt.Printf("%*s<%s>\n", depth*2, "", n.Data)

depth++

}

func endElement(n *html.Node) {

if n.Type == html.ElementNode {

depth--

fmt.Printf("%*s</%s>\n", depth*2, "", n.Data)

}

Thefunctions als o indentthe out put usinganother fmt.Printf tr ick.The * adverb in %*s

pr intsastr ing padde d with a var iable numberofspaces. Thewidth and the str ing are

prov ide d by the arguments depth*2 and "".

If wecal l forEachNode on anHTMLdocument, like this:

forEachNode(doc, startElement, endElement)

we getamoreelaboratevar iat iononthe out put of our earlier outline prog ram:

$gobuild gopl.io/ch5/outline2

$./outline2 http://gopl.io

<html>

<head>

<meta>

</meta>

<title>

</title>

<style>

</style>

</head>

<body>

<table>

<tbody>

<tr>

<td>

<a>

<img>

</img>

...

Exercis e 5.7: De velop startElement and endElement into a general HTML prett y-print er.

Pr int comment nodes, text nodes, andthe att ribut esofeachelement(<a href='...'>). Use

short for mslike <img/> insteadof <img></img> when an elementhas nochi ldren. Write a

test toens ure thatthe out put can beparsedsuccessf ully. (SeeChapt er11.)

Exercis e 5.8: Mo dif y forEachNode so thatthe pre and post func tions retur n aboole an resu lt

indic atingwhether tocontinuethe traversal. Use ittowrite a function ElementByID with the

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

SECTION 5.6. ANONYMOUS FUNCTIONS 135

fo llowing sig naturethatﬁnd s theﬁrs t HTML elementwit h thespeciﬁed id attr ibute. The

func tionshouldstopthe traversal as soon as a match isfound.

func ElementByID(doc *html.Node, id string) *html.Node

Exercis e 5.9: Wr ite a function expand(s string, f func(string) string) string that

repl aces eachsubst ring ‘‘$foo’’ within s by the text retur ned by f("foo").

5.6. Anonymous Functions

Name d func tions can bedeclare d on lyatthe packagelevel,but wecan use a func tionlit era l to

denot e afunctionvalue wit hin anyexpression. A func tionlit eral iswritt enlikeafunction

de clarat ion, but wit houtanamefol low ing the func ke yword . It isanexpression, andits value

is cal le d an an ony mou s func tion.

Func tionlit eralslet usdeﬁne a functionatits point ofuse.Asanexample, the earlier cal l to

strings.Map canberewritt enas

strings.Map(func(r rune) rune { return r + 1 }, "HAL-9000")

Mo reimp ortantly, functions deﬁne d in thisway haveaccesstothe ent ire lexic al enviro nment,

so the inner functioncan refer tovar iables fro m theenclosingfunction, as thisexample

shows:

gopl.io/ch5/squares

// squares returns a function that returns

// the next square number each time it is called.

func squares() func() int {

var x int

return func() int {

x++

return x * x

}

func main() {

f:=squares()

fmt.Println(f()) // "1"

fmt.Println(f()) // "4"

fmt.Println(f()) // "9"

fmt.Println(f()) // "16"

}

Thefunction squares returnsanother function, oftyp e func() int.Acall to squares cre-

ates a localvar iable x andretur nsananony mou s func tionthat, eachtimeitiscal le d,incre-

ments x andretur nsits squ are . Asecon d call to squares wouldcre ate a secon d var iable x and

return a new anony mou s func tionwhichincrementsthatvar iable.

The squares exampledemon strates thatfunctionvalues arenot justcodebut can havestate.

Theanony mou s inner functioncan accessand updatethe local variables ofthe enclosing

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

136 CHAPTER 5. FUNCTIONS

func tion squares.These hidden variablereferences arewhy weclassif y func tions asreference

typesand why functionvalues arenot comparable.Functionvalues like these are imp le-

ment edusingatechnique cal le d cl osure s,and Goprogrammersoften use thister m forfunc-

tion values.

Here again wesee anexamplewhere the lifet imeofavar iable isnot deter mined byits scope:

thevar iable x existsafter squares hasretur ned wit hin main,eventhoug h x is hidden inside f.

As a som ewhat academic exampleofanony mou s func tions,con sider the pro blem ofcom-

puting a sequence ofcomputerscience coursesthatsat isﬁes the prere quisite requirements of

each one.The prere quisitesare given in the prereqs tablebelow,whichisamapping fro m

each cours e to the listofcours esthatmustbecompleted beforeit.

gopl.io/ch5/toposort

// prereqs maps computer science courses to their prerequisites.

var prereqs = map[string][]string{

"algorithms": {"data structures"},

"calculus": {"linear algebra"},

"compilers": {

"data structures",

"formal languages",

"computer organization",

"data structures": {"discrete math"},

"databases": {"data structures"},

"discrete math": {"intro to programming"},

"formal languages": {"discrete math"},

"networks": {"operating systems"},

"operating systems": {"data structures", "computer organization"},

"programming languages": {"data structures", "computer organization"},

}

Thiskindofpro blem isknown astop olog ical sor ting. Con ceptu ally, the prere quisite

infor mat ionfor msadirec ted graphwit h anodefor eachcours e andedges fro m each cours e to

thecours esthatitdep ends on. Thegraphisacyclic: there isnopat h from a cours e that leads

back to its elf.Wecan comp ute a valid sequence usingdepth-ﬁrs t search through the graph

with thecodebelow :

func main() {

for i, course := range topoSort(prereqs) {

fmt.Printf("%d:\t%s\n", i+1, course)

}

func topoSort(m map[string][]string) []string {

var order []string

seen := make(map[string]bool)

var visitAll func(items []string)

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

SECTION 5.6. ANONYMOUS FUNCTIONS 137

visitAll = func(items []string) {

for _, item := range items {

if !seen[item] {

seen[item] = true

visitAll(m[item])

order = append(order, item)

}

var keys []string

for key := range m {

keys = append(keys, key)

}

sort.Strings(keys)

visitAll(keys)

return order

}

Wh enananony mou s func tionrequires rec ursion, as in thisexample, wemustﬁrs t de clare a

var iable,and thenassig n theanony mou s func tiontothatvar iable.Had these two steps been

combined inthe declarat ion, the functionlit eral wou ldnot bewit hin the scope ofthe var iable

visitAll so itwou ldhavenoway tocal l itself rec ursively:

visitAll := func(items []string) {

// ...

visitAll(m[item]) // compile error: undefined: visitAll

// ...

}

Theout put of the toposort prog ram is shown below.Itisdeter minist ic, an often-desirable

prop ertythatdoesn’talways come for fre e.Here, the values ofthe prereqs mapare slices, not

more maps, sotheir iterat ionorder isdeter minist ic, andwesor ted the keysof prereqs before

making the initial cal lsto visitAll.

1: intro to programming

2: discrete math

3: data structures

4: algorithms

5: linear algebra

6: calculus

7: formal languages

8: computer organization

9: compilers

10: databases

11: operating systems

12: networks

13: programming languages

Let’sretur n to our findLinks example. We’ve mov edthe lin k-ext raction function

links.Extract to its own package, since we’ll use itagain in Chapt er8.Wereplace d the

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

138 CHAPTER 5. FUNCTIONS

visit func tionwit h an anonymou s func tionthatapp end s to the links slice direc tly,and used

forEachNode to handlethe traversal. Since Extract ne e dsonlythe pre func tion, itpasses

nil forthe post argument.

gopl.io/ch5/links

// Package links provides a link-extraction function.

package links

import (

"fmt"

"net/http"

"golang.org/x/net/html"

)

// Extract makes an HTTP GET request to the specified URL, parses

// the response as HTML, and returns the links in the HTML document.

func Extract(url string) ([]string, error) {

resp, err := http.Get(url)

if err != nil {

return nil, err

}

if resp.StatusCode != http.StatusOK {

resp.Body.Close()

return nil, fmt.Errorf("getting %s: %s", url, resp.Status)

}

doc, err := html.Parse(resp.Body)

resp.Body.Close()

if err != nil {

return nil, fmt.Errorf("parsing %s as HTML: %v", url, err)

}

var links []string

visitNode := func(n *html.Node) {

if n.Type == html.ElementNode && n.Data == "a" {

for _, a := range n.Attr {

if a.Key != "href" {

continue

}

link, err := resp.Request.URL.Parse(a.Val)

if err != nil {

continue // ignore bad URLs

}

links = append(links, link.String())

}

forEachNode(doc, visitNode, nil)

return links, nil

}

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

SECTION 5.6. ANONYMOUS FUNCTIONS 139

In ste adofapp endingthe raw href attr ibute value tothe links slice,thisversionparsesitasa

URL rel ative tothe bas e URL ofthe document, resp.Request.URL.The resulting link is in

abs olut e form,suitablefor use inacal l to http.Get.

Craw lingthe web is,atits heart,aproblem ofgraphtraversal. The topoSort exampleshowe d

adepth-ﬁrs t traversal; for our web craw ler,we’ll use bre adt h-ﬁrs t traversal, atleast initial ly. In

Chapter 8, we’ll explore con cur renttraversal.

Thefunctionbelow enc apsulates the ess enceofabre adt h-ﬁrs t traversal. The cal ler prov ides

an initial list worklist of itemstovisit and a functionvalue f to cal l foreachitem. Eachitem

is identiﬁed byastr ing . Thefunction f returnsalistofnew items toapp end tothe wor klist.

The breadthFirst func tionretur nswhenall items havebeenvisit ed. Itmaintainsaset of

st rings toens ure thatnoitemisvisit edtwice.

gopl.io/ch5/findlinks3

// breadthFirst calls f for each item in the worklist.

// Any items returned by f are added to the worklist.

// f is called at most once for each item.

func breadthFirst(f func(item string) []string, worklist []string) {

seen := make(map[string]bool)

for len(worklist) > 0 {

items := worklist

worklist = nil

for _, item := range items {

if !seen[item] {

seen[item] = true

worklist = append(worklist, f(item)...)

}

As weexplained inpassinginChapt er3,the argument ‘‘f(item)...’’ caus esall the items in

thelistretur ned by f to beapp ended tothe wor klist.

In our crawler,items are URLs. The crawl func tionwe’ll sup ply to breadthFirst pr intsthe

URL, extrac ts itslin ks, andretur nsthemsothatthe y to o arevisit ed.

func crawl(url string) []string {

fmt.Println(url)

list, err := links.Extract(url)

if err != nil {

log.Print(err)

}

return list

}

To start the craw ler of f,we’ll use the command-linearguments as the initial URLs.

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

140 CHAPTER 5. FUNCTIONS

func main() {

// Crawl the web breadth-first,

// starting from the command-line arguments.

breadthFirst(crawl, os.Args[1:])

}

Let’scraw l theweb startingfro m https://golang.org.Hereare som e of the resulting lin ks:

$gobuild gopl.io/ch5/findlinks3

$./findlinks3 https://golang.org

https://golang.org/

https://golang.org/doc/

https://golang.org/pkg/

https://golang.org/project/

https://code.google.com/p/go-tour/

https://golang.org/doc/code.html

https://www.youtube.com/watch?v=XCsL89YtqCs

http://research.swtch.com/gotour

https://vimeo.com/53221560

...

Thepro cessend s when allreach ableweb pages havebeencraw ledorthe memory ofthe com-

puterisexhausted.

Exercis e 5.10: Re writ e topoSort to use maps insteadofslices andeliminatethe initial sor t.

Verify thatthe results, thoug h nondeter minist ic, arevalid top olog ical order ings.

Exercis e 5.11: Theins tructor of the linearalgebra course decides thatcalc ulu s is now a

prerequisite. Extendthe topoSort func tiontorep ort cyc les.

Exercis e 5.12: The startElement and endElement func tions in gopl.io/ch5/outline2

(§5.5) share a globalvar iable, depth.Tur n them into anony mou s func tions thatshare a var i-

able localtothe outline func tion.

Exercis e 5.13: Mo dif y crawl to makelocal copies ofthe pages it ﬁnd s,cre ating direc tor ies as

ne cessary.Don’tmakecopies ofpages thatcom e from a dif ferentdom ain. For example, ifthe

or iginalpagecom esfro m golang.org,saveall ﬁles fro m there, but exc lude onesfro m

vimeo.com.

Exercis e 5.14: Us e the breadthFirst func tiontoexplore a dif ferentstr ucture. For example,

youcou lduse the cours e dep endencies fro m the topoSort example(adirec ted graph), the ﬁle

systemhierarchyonyourcomputer(atre e), oralistofbus orsubway routesdow nlo ade d from

your city gov ernment’sweb sit e (an undirec ted graph).

5.6.1. Caveat: Capturing Iteration Variables

In thissec tion,we’ll lookatapit fal l of Go’slexic al scop e rules thatcan cause sur prisingresults.

We urge youtounderst and the pro blem beforepro ceeding, because the trap can ensnareeven

exp erience d prog rammers.

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

SECTION 5.6. ANONYMOUS FUNCTIONS 141

Consider a program thatmustcre ate a set ofdirec tor ies andlater remov e them. Wecan use a

slice offunctionvalues toholdthe cle an-upoperat ions.(Fo r brevit y,wehaveomitt edall error

hand ling in thisexample.)

var rmdirs []func()

for _, d := range tempDirs() {

dir := d // NOTE: necessary!

os.MkdirAll(dir, 0755) // creates parent directories too

rmdirs = append(rmdirs, func() {

os.RemoveAll(dir)

})

}

// ...do some work...

for _, rmdir := range rmdirs {

rmdir() // clean up

}

Yo u maybewon der ingwhy weassig ned the loopvar iable d to a new local variable dir within

theloopbody, ins teadofjustnamingthe loopvar iable dir as in thissubtlyincor rec t var iant:

var rmdirs []func()

for _, dir := range tempDirs() {

os.MkdirAll(dir, 0755)

rmdirs = append(rmdirs, func() {

os.RemoveAll(dir) // NOTE: incorrect!

})

}

Thereasonisacon sequence ofthe scope rules for loopvar iables. Inthe program immediate ly

ab ove , the for lo opint roduces a new lexic al blockinwhichthe var iable dir is declare d.All

func tionvalues create d by thisloop ‘‘capt ure’’ andshare the samevar iable—an addressable

storagelocat ion, not its value at thatpar tic ularmom ent.The value of dir is updated insuc-

cessive iterat ions,sobythe timethe cle anup functions are cal le d,the dir var iable has been

up dated several times bythe now-comp leted for lo op. Thu s dir holdsthe value fro m the

ﬁnaliterat ion, andcon sequentlyall cal lsto os.RemoveAll wi l l attemp t to remov e thesame

direc tor y.

Fr equently, the inner variableint roduce d to wor k around thispro blem—dir in our example—

is given the exac t same nameasthe out ervar iable ofwhichitisacopy, leadingtoodd-lo oking

butcruci al var iable decl arat ions likethis:

for _, dir := range tempDirs() {

dir := dir // declares inner dir, initialized to outer dir

// ...

}

Theriskisnot unique to range-b ased for lo ops. Theloop in the examplebelow suf fersfro m

thesamepro blem due tounintended capture ofthe index variable i.

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

142 CHAPTER 5. FUNCTIONS

var rmdirs []func()

dirs := tempDirs()

for i := 0; i < len(dirs); i++ {

os.MkdirAll(dirs[i], 0755) // OK

rmdirs = append(rmdirs, func() {

os.RemoveAll(dirs[i]) // NOTE: incorrect!

})

}

Thepro blem ofiterat ionvar iable capture ismostoften encountere d when usingthe go st ate-

ment (Chapter8)orwit h defer (w hichwewill see inamom ent)since bot h maydel aythe

exec ution of a functionvalue until after the loophas ﬁnishe d.But the pro blem isnot inherent

to go or defer.

5.7. Variadic Functions

A variadic fun cti on is one thatcan becal le d with var yingnumbers ofarguments. Themost

fami liar examples are fmt.Printf andits variants. Printf re quires one ﬁxe d argumentatthe

beginning, thenaccepts anynumberofsubsequentarguments.

To d eclare a var iadic function, the typ e of the ﬁnalparameter ispre ceded byanellipsis, ‘‘...’’,

whichindic ates thatthe functionmay becal le d with any numberofarguments ofthistyp e.

gopl.io/ch5/sum

func sum(vals ...int) int {

total := 0

for _, val := range vals {

total += val

}

return total

}

The sum func tionabove retur nsthe sum ofzeroormore int arguments. Wit hin the bodyof

thefunction, the typ e of vals is an []int slice.When sum is cal le d,any numberofvalues

maybeprovide d forits vals parameter.

fmt.Println(sum()) // "0"

fmt.Println(sum(3)) // "3"

fmt.Println(sum(1, 2, 3, 4)) // "10"

Implicitly, the cal ler al locatesanarray,copies the arguments into it, andpassesaslice ofthe

entire array tothe function. Thelastcal l ab ove thu s behavesthe sameasthe cal l below, which

shows how toinv oke a var iadic functionwhenthe arguments arealready inaslice: place an

el lipsisafter the ﬁnalargument.

values := []int{1, 2, 3, 4}

fmt.Println(sum(values...)) // "10"

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

SECTION 5.8. DEFERRED FUNCTION CALLS 143

Although the ...int parameter beh aveslikea slice wit hin the functionbody, the typ e of a

var iadic functionisdistinc t from the typ e of a functionwit h an ordinar y slice parameter.

func f(...int) {}

func g([]int) {}

fmt.Printf("%T\n", f) // "func(...int)"

fmt.Printf("%T\n", g) // "func([]int)"

Variadic functions are often usedfor str ing for matting . The errorf func tionbelow con-

st ruc ts afor matte d er ror message wit h alinenumberatthe beg inning. The sufﬁx f is a widely

fo llowe d namingconvent ion for var iadic functions thatacceptaPrintf-sty lefor mat str ing .

func errorf(linenum int, format string, args ...interface{}) {

fmt.Fprintf(os.Stderr, "Line %d: ", linenum)

fmt.Fprintf(os.Stderr, format, args...)

fmt.Fprintln(os.Stderr)

}

linenum, name := 12, "count"

errorf(linenum, "undefined: %s", name) // "Line 12: undefined: count"

The interface{} type means thatthisfunctioncan acceptany values at allfor itsﬁnalargu-

ments, as we’ll explain in Chapt er7.

Exercis e 5.15: Wr ite var iadic functions max and min,analogou s to sum.Whatshouldthese

func tions dowhencal le d with noarguments? Write var iants thatrequireatleast one argu-

ment.

Exercis e 5.16: Wr ite a var iadic versionof strings.Join.

Exercis e 5.17: Wr ite a var iadic function ElementsByTagName that, given an HTML nodetre e

andzeroormorenames, retur nsall the elements thatmatch one ofthose names. Hereare two

examplecal ls:

func ElementsByTagName(doc *html.Node, name ...string) []*html.Node

images := ElementsByTagName(doc, "img")

headings := ElementsByTagName(doc, "h1", "h2", "h3", "h4")

5.8. Deferred Function Calls

Our findLinks examples usedthe out put of http.Get as the inp utto html.Parse.This

workswel l if the content ofthe requeste d URL isindeed HTML, but manypages cont ain

images, plain text, andother ﬁle for mats. Feedingsuch ﬁles into anHTMLparsercou ldhave

undesirable effe cts.

Theprogram below fetch esanHTMLdocumentand printsits tit le. The title func tion

insp ectsthe Content-Type he ader ofthe ser ver’srespons e andretur nsanerror ifthe doc-

umentisnot HTML.

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

144 CHAPTER 5. FUNCTIONS

gopl.io/ch5/title1

func title(url string) error {

resp, err := http.Get(url)

if err != nil {

return err

}

// Check Content-Type is HTML (e.g., "text/html; charset=utf-8").

ct := resp.Header.Get("Content-Type")

if ct != "text/html" && !strings.HasPrefix(ct, "text/html;") {

resp.Body.Close()

return fmt.Errorf("%s has type %s, not text/html", url, ct)

}

doc, err := html.Parse(resp.Body)

resp.Body.Close()

if err != nil {

return fmt.Errorf("parsing %s as HTML: %v", url, err)

}

visitNode := func(n *html.Node) {

if n.Type == html.ElementNode && n.Data == "title" &&

n.FirstChild != nil {

fmt.Println(n.FirstChild.Data)

}

forEachNode(doc, visitNode, nil)

return nil

}

Here’satypic al session, slig htlyedite d to ﬁt:

$gobuild gopl.io/ch5/title1

$./title1 http://gopl.io

The Go Programming Language

$./title1 https://golang.org/doc/effective_go.html

Effective Go - The Go Programming Language

$./title1 https://golang.org/doc/gopher/frontpage.png

title: https://golang.org/doc/gopher/frontpage.png

has type image/png, not text/html

Observethe dup lic ated resp.Body.Close() call,whichens uresthat title clos es thenet-

work conne ction on all exe cut ion pat hs, includingfai lures. Asfunctions growmorecomplex

andhavetohandlemoreerror s,suchdup lic ationofcle an-uplog ic maybecom e amain-

tenancepro blem. Let’ssee how Go’snov el defer me chanism makes things simpler.

Sy ntactic ally,adefer st atement isanordinar y func tionormet hod cal l preﬁxe d by the

ke yword defer.The functionand argumentexpressions are evaluate d when the statement is

exec ute d, butthe actualcal l is deferred until the functionthatcontainsthe defer st atement

has ﬁnishe d,whether nor mal ly, byexe cut ing a retur n st atement orfal lingoff the end,or

abnormal ly, bypanicking . Anynumberofcal lsmay bedefer red;the y areexe cut edinthe

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

SECTION 5.8. DEFERRED FUNCTION CALLS 145

re verse ofthe order in whichthe y were defer red.

A defer st atement isoften usedwit h paired operat ions likeopenand clos e,conne ctand dis-

connec t,orlockand unlocktoens ure thatres ources arereleasedinall cas es, no matterhow

comp lex the control ﬂow.The rig htplace for a defer st atement thatreleasesares ource is

immediate lyafter the res ource has beensuccessf ullyacquired.Inthe title func tionbelow,a

singledefer red cal l repl aces bot h previous cal lsto resp.Body.Close():

gopl.io/ch5/title2

func title(url string) error {

resp, err := http.Get(url)

if err != nil {

return err

}

defer resp.Body.Close()

ct := resp.Header.Get("Content-Type")

if ct != "text/html" && !strings.HasPrefix(ct, "text/html;") {

return fmt.Errorf("%s has type %s, not text/html", url, ct)

}

doc, err := html.Parse(resp.Body)

if err != nil {

return fmt.Errorf("parsing %s as HTML: %v", url, err)

}

// ...print doc'stitle element...

return nil

}

Thesamepattern can beusedfor other res ources beside net wor k connec tion s,for ins tance to

clos e an open ﬁle:

io/ioutil

package ioutil

func ReadFile(filename string) ([]byte, error) {

f, err := os.Open(filename)

if err != nil {

return nil, err

}

defer f.Close()

return ReadAll(f)

}

or tounlockamut ex(§9.2):

var mu sync.Mutex

var m = make(map[string]int)

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

146 CHAPTER 5. FUNCTIONS

func lookup(key string) int {

mu.Lock()

defer mu.Unlock()

return m[key]

}

The defer st atement can also beusedtopair ‘‘on ent ry’’ and ‘‘on exit’’ ac tions whendebug ging

acomplex function. The bigSlowOperation func tionbelow cal ls trace immediate ly, which

do es the ‘‘on ent ry’’ ac tionthenretur nsafunctionvalue that, whencal le d,doesthe cor-

resp onding ‘‘on exit’’ ac tion. Bydefer r ingacal l to the retur ned functioninthisway,wecan

inst rumentthe ent rypoint and all exitpointsofa functioninasinglestatement and evenpass

values, like the start time,bet weenthe two actions.But don’tfor g etthe ﬁnalparenthes es in

the defer st atement,orthe ‘‘on ent ry’’ ac tionwill happenonexitand the on-exitactionwon’t

happ en at all!

gopl.io/ch5/trace

func bigSlowOperation() {

defer trace("bigSlowOperation")() // don'tforget the extra parentheses

// ...lots of work...

time.Sleep(10 * time.Second) // simulate slow operation by sleeping

}

func trace(msg string) func() {

start := time.Now()

log.Printf("enter %s", msg)

return func() { log.Printf("exit %s (%s)", msg, time.Since(start)) }

}

Each time bigSlowOperation is cal le d,itlogs its entr y andexitand the elaps edtimebet ween

them. (Weused time.Sleep to simulateaslowoperat ion.)

$gobuild gopl.io/ch5/trace

$./trace

2015/11/18 09:53:26 enter bigSlowOperation

2015/11/18 09:53:36 exit bigSlowOperation (10.000589217s)

Deferred functions run af ter return statementshaveupdated the function’sresultvar iables.

Becaus e an anonymou s func tioncan accessits enclosingfunction’svar iables, includingnamed

resu lts, a defer red anony mou s func tioncan obs erve the function’sresults.

Consider the function double:

func double(x int) int {

return x + x

}

By namingits resultvar iable andaddingadefer st atement,wecan makethe functionprint its

arguments andresults eachtimeitiscal le d.

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

SECTION 5.8. DEFERRED FUNCTION CALLS 147

func double(x int) (result int) {

defer func() { fmt.Printf("double(%d) = %d\n", x, result) }()

return x + x

}

_=double(4)

// Output:

// "double(4) = 8"

Thistrick isoverk ill for a functionassimpleas double butmay beusefulinfunctions wit h

many retur n st atements.

Adefer red anony mou s func tioncan evenchange the values thatthe enclosingfunction

returnstoits cal ler:

func triple(x int) (result int) {

defer func() { result += x }()

return double(x)

}

fmt.Println(triple(4)) // "12"

Becaus e defer red functions aren’texe cut edunt i l thever y endofafunction’sexe cut ion,a

defer st atement inaloopdes ervesext ra scrutiny.The codebelow cou ldrun out of ﬁle

des crip tor s since no ﬁle will beclos edunt i l al l ﬁles havebeenpro cessed:

for _, filename := range filenames {

f, err := os.Open(filename)

if err != nil {

return err

}

defer f.Close() // NOTE: risky; could run out of file descriptors

// ...process f...

}

Onesolut ion istomov e theloopbody, includingthe defer st atement,int o anot her function

that iscal le d on eachiterat ion.

for _, filename := range filenames {

if err := doFile(filename); err != nil {

return err

}

func doFile(filename string) error {

f, err := os.Open(filename)

if err != nil {

return err

}

defer f.Close()

// ...process f...

}

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

148 CHAPTER 5. FUNCTIONS

Theexamplebelow is an imp rov ed fetch prog ram (§1.5) thatwritesthe HTTPrespons e to a

lo cal ﬁle insteadoftothe stand ard out put.Itder ivesthe ﬁle namefro m thelastcomponent of

theURL pat h, whichitobt ainsusingthe path.Base func tion.

gopl.io/ch5/fetch

// Fetch downloads the URL and returns the

// name and length of the local file.

func fetch(url string) (filename string, n int64, err error) {

resp, err := http.Get(url)

if err != nil {

return "", 0, err

}

defer resp.Body.Close()

local := path.Base(resp.Request.URL.Path)

if local == "/" {

local = "index.html"

}

f, err := os.Create(local)

if err != nil {

return "", 0, err

}

n, err = io.Copy(f, resp.Body)

// Close file, but prefer error from Copy, if any.

if closeErr := f.Close(); err == nil {

err = closeErr

}

return local, n, err

}

Thedefer red cal l to resp.Body.Close shouldbefami liar bynow.It’stempt ing touse a

second defer red cal l,to f.Close,toclos e thelocal ﬁle,but thiswou ldbesubtlywro ngbecause

os.Create op ens a ﬁle for writing , creating itasneeded.Onmany ﬁle systems,not ably NFS,

wr ite error s arenot rep orted immediate lybut may bepostp one d until the ﬁle isclos ed. Fai l-

uretoche ckthe resultofthe clos e op erat ioncou ldcause seriou s data losstogounnot ice d.

Ho wever,ifbot h io.Copy and f.Close fai l,weshouldprefertorep ort the error fro m

io.Copy since itocc urre d ﬁrs t andismorelikelytotel l us the rootcause.

Exercis e 5.18: Wi thout chang ing its beh avior,rewrite the fetch func tiontouse defer to clos e

thewritable ﬁle.

5.9. Panic

Go’styp e systemcatch esmanymistakesatcompi letime, but others, like anout-of-b ounds

ar ray accessornil point erdereference,requireche cks at run time. Whenthe Goruntime

detec tsthese mistakes, it pani cs.

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

SECTION 5.9. PANIC 149

During a typic al panic, nor mal exec ution stops, alldefer red functioncal lsinthatgoroutine are

exec ute d, andthe program crasheswit h alog message . Thislog message includes the pani c

valu e,whichisusu allyanerror message ofsom e sort,and,for eachgoroutine,asta cktra c e

showing the stack offunctioncal lsthatwereactive at the timeofthe panic. Thislog message

of ten has enough infor mat iontodiagnos e therootcause ofthe pro blem wit houtrunningthe

prog ram again, soitshouldalways beinclude d in a bug rep ort about a panicking program.

No t al l panics come fro m theruntime.The bui lt-in panic func tionmay becal le d direc tly ; it

accepts anyvalue as an argument. A panic isoften the bestthingtodowhensom e ‘‘impossi-

ble’’ situ ation happens,for ins tance,exe cut ion reach esacas e that logic ally can’thappen:

switch s := suit(drawCard()); s {

case "Spades": // ...

case "Hearts": // ...

case "Diamonds": // ...

case "Clubs": // ...

default:

panic(fmt.Sprintf("invalid suit %q", s)) // Joker?

}

It’sgood prac tice toass ert thatthe pre con dit ion s of a functionhold, but thiscan easi lybedon e

to excess. Unlessyou can provide a moreinfor mat ive error message ordetec t an erro r so oner,

thereisnopoint ass ertingacon dit ion thatthe runtime will che ckfor you .

func Reset(x *Buffer) {

if x == nil {

panic("x is nil") // unnecessary!

}

x.elements = nil

}

Although Go’spanic mechanism res embles exception s in other langu ages, the sit uat ions in

whichpanic isusedare quite dif ferent. Since a panic causesthe program tocrash, itisgener-

al lyusedfor graveerror s,such as a log ical inconsistenc y in the program; diligentprogram-

mers con sider anycrashtobepro ofofabug in their code.Inarobustprogram, ‘‘exp ected’’

er ror s,the kindthatarise fro m incorrec t input, misconﬁgurat ion, orfai lingI/O,shouldbe

hand led gracef ully; the y arebestdealt wit h using error values.

Consider the function regexp.Compile,whichcompi les a regu lar expressionint o an efﬁcient

form for matching. Itretur nsan error if cal le d with anill-for med patt ern,but che cking this

er ror isunnecessary and burdensome ifthe cal ler knowsthatapartic ularcal l cannot fai l.In

such cas es, it’sreasonablefor the cal ler to handleanerror bypanicking , since itisbelie ved to

be imp ossible.

Sincemostregu lar expressions are lit eralsinthe program sourcecode, the regexp package

prov ides a wrapperfunction regexp.MustCompile that does thische ck:

package regexp

func Compile(expr string) (*Regexp, error) { /* ... */ }

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

150 CHAPTER 5. FUNCTIONS

func MustCompile(expr string) *Regexp {

re, err := Compile(expr)

if err != nil {

panic(err)

}

return re

}

Thewrapperfunctionmakes itconvenientfor clients toinitialize a package-le vel var iable wit h

acompi led regu lar expression, like this:

var httpSchemeRE = regexp.MustCompile(`^https?:`)//"http:" or "https:"

Of course, MustCompile shouldnot becal le d with unt ruste d inputvalues. The Must preﬁxisa

common namingconvent ion for functions ofthiskind, like template.Must in Sec tion 4.6.

Wh enapanic occ urs, alldefer red functions are run in reverse order,startingwit h thos e of the

topm ost functiononthe stack and pro ceedingupto main,asthe program below

demon strates:

gopl.io/ch5/defer1

func main() {

f(3)

}

func f(x int) {

fmt.Printf("f(%d)\n", x+0/x) // panics if x == 0

defer fmt.Printf("defer %d\n", x)

f(x - 1)

}

Wh enrun, the program printsthe fol low ing tothe stand ard out put:

f(3)

f(2)

f(1)

defer 1

defer 2

defer 3

Apanic occ ursdur ingthe cal l to f(0),causingthe three defer red cal lsto fmt.Printf to run.

Then the runtime ter minates the program, print ing the panic message and a stack dumpto

thestand ard error stream(simpliﬁed for clarity):

panic: runtime error: integer divide by zero

main.f(0)

src/gopl.io/ch5/defer1/defer.go:14

main.f(1)

src/gopl.io/ch5/defer1/defer.go:16

main.f(2)

src/gopl.io/ch5/defer1/defer.go:16

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

SECTION 5.10. RECOVER 151

main.f(3)

src/gopl.io/ch5/defer1/defer.go:16

main.main()

src/gopl.io/ch5/defer1/defer.go:10

As wewill see soon, itispossiblefor a functiontorecov erfro m apanic sothatitdoesnot ter-

minatethe program.

Fo r di agnost icpur pos es, the runtime packagelets the programmer dumpthe stack usingthe

same machiner y.Bydefer r ingacal l to printStack in main,

gopl.io/ch5/defer2

func main() {

defer printStack()

f(3)

}

func printStack() {

var buf [4096]byte

n:=runtime.Stack(buf[:], false)

os.Stdout.Write(buf[:n])

}

thefol low ing addition altext (again simpliﬁed for clarity)isprint edtothe stand ard out put:

goroutine 1 [running]:

main.printStack()

src/gopl.io/ch5/defer2/defer.go:20

main.f(0)

src/gopl.io/ch5/defer2/defer.go:27

main.f(1)

src/gopl.io/ch5/defer2/defer.go:29

main.f(2)

src/gopl.io/ch5/defer2/defer.go:29

main.f(3)

src/gopl.io/ch5/defer2/defer.go:29

main.main()

src/gopl.io/ch5/defer2/defer.go:15

Re adersfami liar wit h exception s in other langu ages may besur prisedthat runtime.Stack

canprint infor mat ionabout functions thatseemtohavealready been ‘‘unwound.’’ Go’spanic

me chanism runsthe defer red functions before it unw ind s thestack.

5.10. Recover

Gi vingupisusu allythe rig htrespons e to a panic, but not always. Itmig htbepossibleto

re cov erinsom e way,oratleast cle an up the mess beforequitt ing . Fo r example, a web ser ver

that encounters anunexp ected pro blem couldclos e theconne ction rat her thanleave the client

hang ing , anddur ingdevelopment,itmig htrep ort the error tothe clienttoo.

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

152 CHAPTER 5. FUNCTIONS

If the bui lt-in recover func tioniscal le d within a defer red functionand the functioncontain-

ingthe defer st atement ispanicking , recover ends the cur rentstate ofpanic andretur nsthe

panic value.The functionthatwas panicking doesnot continuewhere itlef t of f butretur ns

normal ly. If recover is cal le d at anyother time, ithas noeffec t andretur ns nil.

To illustrate, con sider the development ofaparserfor a langu age. Evenwhenitapp earstobe

work ing well,given the complexity ofits job,bugs may still lurkinobs curecor ner cas es. We

mig htpreferthat, insteadofcrashing, the parsertur nsthese panics into ordinar y pars e er ror s,

perh aps wit h an extra message exhor tingthe userto ﬁle a bug rep ort.

func Parse(input string) (s *Syntax, err error) {

defer func() {

if p := recover(); p != nil {

err = fmt.Errorf("internal error: %v", p)

}

}()

// ...parser...

}

Thedefer red functionin Parse re cov ers fro m apanic, usingthe panic value tocon str uct an

er ror message;afancier versionmig htinclude the ent ire cal l st ack using runtime.Stack.The

defer red functionthenassig nstothe err resu lt, whichisretur ned tothe cal ler.

Re cov ering indis criminatelyfro m panics isadubious prac tice because the state ofapackage’s

var iables afterapanic israrelywel l deﬁne d or documente d. Perh aps a crit icalupdatetoadat a

st ruc turewas incomp lete, a ﬁle ornet wor k connec tion was opene d butnot clos ed, oralock

was acquired but not released. Fur thermore, byreplacingacrashwit h,say,aline in a log ﬁle,

indiscriminaterecov ery may cause bugs togounnot ice d.

Re cov ering fro m apanic wit hin the samepackagecan helpsimplif y thehandlingofcomplex

or unexp ected error s,but asageneral rule, you shouldnot att emp t to recov erfro m anot her

package’spanic. Public APIs shouldrep ort fai lures as errors. Simi larly,you shouldnot

re cov erfro m apanic thatmay passthrough a functionyou donot maintain, suchasacal ler-

prov ide d callback,since you cannot reasonabout itssafet y.

Fo r example, the net/http packageprovides a web ser ver thatdispatch esincomingrequests

to user-prov ide d hand ler functions.Rat her thanlet a panic in one ofthese handlerskill the

process, the ser ver cal ls recover,printsastack trace,and continues ser ving. Thisiscon-

venientinprac tice,but it doesriskleaking res ources orleaving the fai le d hand ler in an

unsp eciﬁed state thatcou ldleadtoother pro blems.

Fo r al l theabove reasons,it’ssafesttorecov erselec tive lyifatall.Inother words,recov eronly

from panics thatwereint ended toberecov ere d from,whichshouldberare. Thisint ent ion

canbeencoded byusingadist inc t,unexp orted typ e forthe panic value andtesting whether

thevalue retur ned by recover hasthattyp e.(We’ll see one way todothisinthe next exam-

ple.)Ifso, werep ort the panic as an ordinar y error;ifnot,wecal l panic with thesamevalue

to resumethe state ofpanic.

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

SECTION 5.10. RECOVER 153

Theexamplebelow isavar iat iononthe title prog ram that rep ortsanerror ifthe HTML

do cumentcontainsmultiple <title> elements. Ifso, itabortsthe rec ursionbycal ling panic

with a value ofthe speci al type bailout.

gopl.io/ch5/title3

// soleTitle returns the text of the first non-empty title element

// in doc, and an error if there was not exactly one.

func soleTitle(doc *html.Node) (title string, err error) {

type bailout struct{}

defer func() {

switch p := recover(); p {

case nil:

// no panic

case bailout{}:

// "expected" panic

err = fmt.Errorf("multiple title elements")

default:

panic(p) // unexpected panic; carry on panicking

}

}()

// Bail out of recursion if we find more than one non-empty title.

forEachNode(doc, func(n *html.Node) {

if n.Type == html.ElementNode && n.Data == "title" &&

n.FirstChild != nil {

if title != "" {

panic(bailout{}) // multiple title elements

}

title = n.FirstChild.Data

}

}, nil)

if title == "" {

return "", fmt.Errorf("no title element")

}

return title, nil

}

Thedefer red handler functioncal ls recover,che cks the panic value,and rep ortsanordinar y

er ror ifthe value was bailout{}.All other non-ni l values indic ateanunexp ected panic, in

whichcas e thehandler cal ls panic with that value,undoing the effec t of recover andresum-

ingthe originalstate ofpanic. (Thisexampledoessom ewhat violate our advice aboutnot

usingpanics for ‘‘exp ected’’ er ror s,but it prov ides a comp act illustrat ionofthe mech anics.)

Fr omsom e condit ion s thereisnorecov ery.Runningout of memor y,for example, causesthe

Go runtime toter minatethe program wit h afat al er ror.

Exercis e 5.19: Us e panic and recover to write a functionthatcontainsno return st atement

yetretur nsanon-zerovalue.

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

This page intentionally left blank

From the Library of YIGUANG HU

ptg16105617

Me thods

Sincethe early 1990s, obj e ct-oriente d prog ramming(OOP) has beenthe dominantprogram-

mingparadig m in indu str y andeducation,and nearly all widely usedlangu ages deve lop ed

since thenhaveinclude d supp ort for it.Goisnoexception.

Although there isnounivers allyaccepte d deﬁnition of obj e ct-oriente d prog ramming, for our

purpos es, an ob jec t is simply a value orvar iable thathas met hods, andametho d is a function

asso ciate d with a par tic ulartyp e.Anobj e ct-oriente d prog ram is one thatusesmet hodsto

express the pro per ties andoperat ions ofeachdat a st ruc turesothatclients need not accessthe

objec t’srepresent ation direc tly.

In earlier chapt ers,wehavemade regu lar use ofmet hodsfro m thestand ard librar y,likethe

Seconds method oftyp e time.Duration:

const day = 24 * time.Hour

fmt.Println(day.Seconds()) // "86400"

andwedeﬁne d amet hod ofour own inSec tion 2.5, a String method for the Celsius type:

func (c Celsius) String() string { return fmt.Sprintf("%g°C", c) }

In thischapt er, the ﬁrs t of two onobj e ct-oriente d prog ramming, we’ll showhow todeﬁne and

us e methodseffec tive ly. We’ll als o covertwo key princip les ofobj e ct-oriente d prog ramming,

encapsul ati on and comp ositi on.

6.1. Method Declarations

Amet hod isdeclare d with a var iantofthe ordinar y func tiondeclarat ioninwhichanext ra

parameter appearsbeforethe functionname. The parameter attach esthe functiontothe typ e

of thatparameter.

155

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

156 CHAPTER 6. METHODS

Let’swrite our ﬁrs t method inasimplepackagefor plane geometr y:

gopl.io/ch6/geometry

package geometry

import "math"

type Point struct{ X, Y float64 }

// traditional function

func Distance(p, q Point) float64 {

return math.Hypot(q.X-p.X, q.Y-p.Y)

}

// same thing, but as a method of the Point type

func (p Point) Distance(q Point) float64 {

return math.Hypot(q.X-p.X, q.Y-p.Y)

}

Theext ra parameter p is cal le d themet hod’s re ceiver,alegac y from early obj e ct-oriente d lan-

gu ages thatdes crib edcal lingamet hod as ‘‘sendingamessage to an obj e ct.’’

In Go, wedon’tuse a speci al name like this or self forthe receiver; wecho ose receiver

namesjustaswewou ldfor any other parameter.Since the receivernamewill befre quently

us ed, it’sagood ide a to choose somethingshort and tobecon sistent acrossmet hods. A com-

monchoice isthe ﬁrs t letterofthe typ e name,like p for Point.

In a met hod cal l,the receiverargumentapp earsbeforethe method name. Thisparal lels the

de clarat ion, in whichthe receiverparameter appearsbeforethe method name.

p:=Point{1, 2}

q:=Point{4, 6}

fmt.Println(Distance(p, q)) // "5", function call

fmt.Println(p.Distance(q)) // "5", method call

There’snoconﬂic t betweenthe two declarat ions offunctions cal le d Distance ab ove . The ﬁrs t

de claresapackage-le vel functioncal le d geometry.Distance.The secon d de claresamet hod

of the typ e Point,soits nameis Point.Distance.

Theexpression p.Distance is cal le d a selector,because itselec tsthe appro priate Distance

method for the receiver p of typ e Point.Selec tor s areals o us edtoselec t ﬁelds ofstr uct typ es,

as in p.X.Since met hodsand ﬁelds inhabit the samenamespace,declaring a met hod X on the

st ruc t type Point wouldbeambiguous and the compi ler will rej e ctit.

Sinceeachtyp e hasits own namespace for met hods, wecan use the name Distance forother

methodssolon g as the y belong todif ferenttyp es. Let’sdeﬁne a typ e Path that representsa

sequence oflineseg ments andgiveitaDistance method too.

// A Path is a journey connecting the points with straight lines.

type Path []Point

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

SECTION 6.1. METHOD DECLARATIONS 157

// Distance returns the distance traveled along the path.

func (path Path) Distance() float64 {

sum := 0.0

for i := range path {

if i > 0 {

sum += path[i-1].Distance(path[i])

}

return sum

}

Path is a name d slice typ e,not a str uct typ e li ke Point,yet wecan still deﬁne methodsfor it.

In allow ing met hodstobeass oci ated wit h anytyp e,Goisunlikemanyother obj e ct-oriente d

languages. Itisoften conv enienttodeﬁne addition albeh avior s forsimpletyp es such asnum-

bers,str ings, slices, maps, andsom etimesevenfunctions.Met hodsmay bedeclare d on any

name d type deﬁne d in the samepackage, solon g as its underly ing typ e is neither a point ernor

an interface.

Thetwo Distance methodshavedif ferenttyp es. They’re not rel ate d to eachother at all,

though Path.Distance us es Point.Distance internal lytocompute the lengt h of eachseg-

ment thatconne cts adjacentpoints.

Let’scal l thenew met hod tocompute the per imeter ofarig httriangle:

perim := Path{

{1, 1},

{5, 1},

{5, 4},

{1, 1},

}

fmt.Println(perim.Distance()) // "12"

In the two cal lsabove tomet hodsnamed Distance,the compi ler deter mines whichfunction

to cal l basedonbot h themet hod nameand the typ e of the receiver. Inthe ﬁrs t, path[i-1]

hastyp e Point so Point.Distance is cal le d; in the secon d, perim hastyp e Path,so

Path.Distance is cal le d.

Al l methodsofagiven typ e mu sthaveunique names, but dif ferenttyp es canuse the same

name for a met hod,likethe Distance methodsfor Point and Path;there’snoneed toqualif y

func tionnames (for example, PathDistance)todis ambigu ate. Herewesee the ﬁrs t beneﬁtto

usingmet hodsoverordinar y func tions:met hod names can beshorter.The beneﬁtismag ni-

ﬁed for cal lsoriginating outside the package, since the y canuse the shorter name an d omit the

packagename:

import "gopl.io/ch6/geometry"

perim := geometry.Path{{1, 1}, {5, 1}, {5, 4}, {1, 1}}

fmt.Println(geometry.PathDistance(perim)) // "12", standalone function

fmt.Println(perim.Distance()) // "12", method of geometry.Path

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

158 CHAPTER 6. METHODS

6.2. Methods with a PointerReceiver

Becaus e callingafunctionmakes a copy ofeachargumentvalue,ifafunctionneedstoupdate

avar iable,orifanargumentissolarge thatwewishtoavoid copy ing it, wemustpassthe

addressofthe var iable usingapoint er. The samegoesfor met hodsthatneed toupdatethe

re ceivervar iable: weatt ach themtothe point ertyp e,suchas *Point.

func (p *Point) ScaleBy(factor float64) {

p.X *= factor

p.Y *= factor

}

Thenameofthismet hod is (*Point).ScaleBy.The parenthes es arenecessary ; without

them, the expressionwou ldbeparsedas *(Point.ScaleBy).

In a realist icprogram, conv ent ion dic tates that if any met hod of Point hasapoint erreceiver,

then al l methodsof Point shouldhaveapoint erreceiver, evenonesthatdon’tstr ictly need it.

We’ve bro ken thisrulefor Point so thatwecan showbot h kind s of met hod.

Name d types(Point)and point ers to them(*Point)are the onlytyp es that may appear in a

re ceiverdeclarat ion. Fur thermore, to avoid ambiguities, met hod declarat ions are not per mit-

te d on named typ es that arethems elves point ertyp es:

type P *int

func (P) f() { /* ... */ } // compile error: invalid receiver type

The (*Point).ScaleBy method can becal le d by providinga*Point re ceiver, likethis:

r:=&Point{1, 2}

r.ScaleBy(2)

fmt.Println(*r) // "{2, 4}"

or this:

p:=Point{1, 2}

pptr := &p

pptr.ScaleBy(2)

fmt.Println(p) // "{2, 4}"

or this:

p:=Point{1, 2}

(&p).ScaleBy(2)

fmt.Println(p) // "{2, 4}"

Butthe lasttwo cas es areungain ly. For tunately, the langu agehelps ushere. Ifthe receiver p is

a variab le of typ e Point butthe method requires a *Point re ceiver, wecan use thisshorthand:

p.ScaleBy(2)

andthe compi ler will per for m an implicit &p on the var iable.Thiswor ksonlyfor var iables,

includingstr uct ﬁelds like p.X andarray orslice elements like perim[0].Wecannot cal l a

*Point method onanon-addressable Point re ceiver, because there’snoway toobt ain the

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

SECTION 6.2. METHODS WITH A POINTER RECEIVER 159

addressofatemporar y value.

Point{1, 2}.ScaleBy(2) // compile error: can'ttake address of Point literal

Butwe can call a Point method like Point.Distance with a *Point re ceiver, because there is

away toobt ain the value fro m theaddress: justloadthe value point edtobythe receiver. The

comp iler inserts an imp licit * op erat ionfor us. Thes e twofunctioncal lsare equivalent:

pptr.Distance(q)

(*pptr).Distance(q)

Let’ssummar ize thes e thre e casesagain, since the y areafre quentpoint ofconfusion. Inevery

valid met hod cal l expression, exactlyone ofthese three statementsistrue.

Either the receiverargumenthas the sametyp e as the receiverparameter,for examplebot h

have typ e T or bot h have typ e *T:

Point{1, 2}.Distance(q) // Point

pptr.ScaleBy(2) // *Point

Or the receiverargumentisavariab le of typ e T andthe receiverparameter has typ e *T.The

comp iler implicitlytakes the addressofthe var iable:

p.ScaleBy(2) // implicit (&p)

Or the receiverargumenthas typ e *T andthe receiverparameter has typ e T.The compi ler

implicitlydereferences the receiver, inother words,loads the value:

pptr.Distance(q) // implicit (*pptr)

If all the methodsofanamed typ e T have a receivertyp e of T itself (not *T), itissafetocopy

inst ances ofthattyp e; callingany ofits met hodsnecessarily makes a copy.For example,

time.Duration values arelib eral lycopie d,includingasarguments tofunctions.But ifany

method has a point erreceiver, you shouldavoid copy ing ins tances of T becaus e doingsomay

viol ate int ernal invar iants. For example, copying anins tance of bytes.Buffer wouldcause

theoriginaland the copytoali as (§2.3.2) the sameunderly ing array ofbytes. Subsequent

method cal lswou ldhaveunp redic table effe cts.

6.2.1. Nil IsaValid Receiver Value

Ju stassom e func tions allow nil point ers asarguments, sodosom e methodsfor their receiver,

especi ally if nil is a meaningf ul zerovalue ofthe typ e,aswit h maps andslices. Inthissimple

lin ked listofint egers, nil repres ents the emp tylist:

// An IntList is a linked list of integers.

// A nil *IntList represents the empty list.

type IntList struct {

Value int

Tail *IntList

}

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

160 CHAPTER 6. METHODS

// Sum returns the sum of the list elements.

func (list *IntList) Sum() int {

if list == nil {

return 0

}

return list.Value + list.Tail.Sum()

}

Wh enyou deﬁne a typ e whos e methodsallow nil as a receivervalue,it’swor thpoint ing this

outexplicitly in its document ation comment, as we did abov e.

Here’spar t of the deﬁnition of the Values type fro m the net/url package:

net/url

package url

// Values maps a string key to a list of values.

type Values map[string][]string

// Get returns the first value associated with the given key,

// or "" if there are none.

func (v Values) Get(key string) string {

if vs := v[key]; len(vs) > 0 {

return vs[0]

}

return ""

}

// Add adds the value to key.

// It appends to any existing values associated with key.

func (v Values) Add(key, value string) {

v[key] = append(v[key], value)

}

It exp oses its represent ation asamap but als o prov ides met hodstosimplif y accesstothe map,

whos e values areslices ofstr ings—it’samu ltimap.Its clients can use its intr insic operator s

(make,slice literals, m[key],and soon), orits met hods, orbot h, as the y prefer :

gopl.io/ch6/urlvalues

m:=url.Values{"lang": {"en"}} // direct construction

m.Add("item", "1")

m.Add("item", "2")

fmt.Println(m.Get("lang")) // "en"

fmt.Println(m.Get("q")) // ""

fmt.Println(m.Get("item")) // "1" (first value)

fmt.Println(m["item"]) // "[1 2]" (direct map access)

m=nil

fmt.Println(m.Get("item")) // ""

m.Add("item", "3") // panic: assignment to entry in nil map

In the ﬁnalcal l to Get,the nil receiverbeh aveslikeanemp tymap.Wecou ldequivalentlyhave

wr itt enitas Values(nil).Get("item")),but nil.Get("item") wi l l notcompi lebecause

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

SECTION 6.3. COMPOSING TYPES BYSTRUCT EMBEDDING 161

thetyp e of nil hasnot beendeter mined.Bycontrast, the ﬁnalcal l to Add panics as ittries to

up dateanil map.

Becaus e url.Values is a maptyp e andamap referstoits key/value pairsindirec tly,any

up dates anddelet ions that url.Values.Add makestothe map elements arevisible tothe

caller.How ever, aswit h ordinar y func tions,any changesamet hod makes tothe reference

itself,likesetting itto nil or mak ingitrefer toadif ferentmap dat a st ruc ture, will not be

reﬂec ted inthe cal ler.

6.3. Composing Types byStruct Embedding

Consider the typ e ColoredPoint:

gopl.io/ch6/coloredpoint

import "image/color"

type Point struct{ X, Y float64 }

type ColoredPoint struct {

Point

Color color.RGBA

}

We cou ldhavedeﬁne d ColoredPoint as a str uct ofthree ﬁelds,but ins teadwe embedded a

Point to provide the X and Y ﬁelds.Aswesaw in Sec tion 4.4.3, embeddinglets ustakeasyn-

tactic shortcuttodeﬁningaColoredPoint that cont ainsall the ﬁelds of Point,plu s some

more . If wewant, wecan selec t theﬁelds of ColoredPoint that werecontr ibute d by the

embedde d Point withoutmentioning Point:

var cp ColoredPoint

cp.X = 1

fmt.Println(cp.Point.X) // "1"

cp.Point.Y = 2

fmt.Println(cp.Y) // "2"

Asimi lar mechanism applies tothe metho ds of Point.Wecan cal l methodsofthe emb edde d

Point ﬁeldusingareceiveroftyp e ColoredPoint,eventhoug h ColoredPoint hasno

de clare d methods:

red := color.RGBA{255, 0, 0, 255}

blue := color.RGBA{0, 0, 255, 255}

var p = ColoredPoint{Point{1, 1}, red}

var q = ColoredPoint{Point{5, 4}, blue}

fmt.Println(p.Distance(q.Point)) // "5"

p.ScaleBy(2)

q.ScaleBy(2)

fmt.Println(p.Distance(q.Point)) // "10"

Themet hodsof Point have been prom ote d to ColoredPoint.Inthisway,emb eddingallows

comp lex typ es with manymet hodstobebui ltupbythe comp ositi on of several ﬁelds,each

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

162 CHAPTER 6. METHODS

prov idingafew met hods.

Re adersfami liar wit h cl ass-b ased obj e ct-oriente d languagesmay betempt edtovie w Point as

abas e cl ass and ColoredPoint as a sub class order ive d cl ass,ortoint erpretthe rel ation ship

betweenthese typ es as if a ColoredPoint ‘‘is a’’ Point.But thatwou ldbeamistake . No tice

thecal lsto Distance ab ove . Distance hasaparameter oftyp e Point,and q is not a Point,so

although q do es have anemb edde d ﬁeldofthattyp e,wemustexplicitlyselec t it.Att emp ting

to pass q would be an error :

p.Distance(q) // compile error: cannot use q (ColoredPoint) as Point

A ColoredPoint is not a Point,but it ‘‘hasa’’ Point,and ithas two addition almet hods Dis-

tance and ScaleBy prom ote d from Point.Ifyou prefertothin k in ter msofimp lementation,

theemb edde d ﬁeldins tructsthe compi ler togenerateaddition alwrappermet hodsthatdele-

gatetothe declare d methods, equivalenttothese:

func (p ColoredPoint) Distance(q Point) float64 {

return p.Point.Distance(q)

}

func (p *ColoredPoint) ScaleBy(factor float64) {

p.Point.ScaleBy(factor)

}

Wh en Point.Distance is cal le d by the ﬁrs t of these wrappermet hods, its receivervalue is

p.Point,not p,and there isnoway for the method toaccessthe ColoredPoint in whichthe

Point is emb edde d.

Thetyp e of ananony mou s ﬁeldmay beapointer to a named typ e,inwhichcas e ﬁelds and

methodsare pro mot edindirec tly fro m thepoint ed-toobj e ct. Addinganother leve l of indi-

re ction lets usshare commonstr uctures andvar y therel ation ships bet weenobj e cts dynami-

cally. The declarat ionof ColoredPoint belowemb eds a *Point:

type ColoredPoint struct {

*Point

Color color.RGBA

}

p:=ColoredPoint{&Point{1, 1}, red}

q:=ColoredPoint{&Point{5, 4}, blue}

fmt.Println(p.Distance(*q.Point)) // "5"

q.Point = p.Point // p and q now share the same Point

p.ScaleBy(2)

fmt.Println(*p.Point, *q.Point) // "{2 2} {2 2}"

Astr uct typ e mayhavemorethanone anony mou s ﬁeld. Had wedeclare d ColoredPoint as

type ColoredPoint struct {

Point

color.RGBA

}

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

SECTION 6.3. COMPOSING TYPES BYSTRUCT EMBEDDING 163

then a value ofthistyp e wouldhaveall the methodsof Point,all the methodsof RGBA,and

anyaddition almet hodsdeclare d on ColoredPoint direc tly.Whenthe compi ler res olves a

sele ctorsuchas p.ScaleBy to a met hod,itﬁrs t lo oks for a direc tly declare d method named

ScaleBy,thenfor met hodspro mot edoncefro m ColoredPoint’semb edde d ﬁelds,thenfor

methodspro mot edtwice fro m embedde d ﬁelds wit hin Point and RGBA,and soon. Thecom-

pi ler rep ortsanerror ifthe selec tor was ambiguous because two met hodswerepro mot edfro m

thesameran k.

Methodscan bedeclare d on lyonnamed typ es (li ke Point)and point ers to them(*Point),

butthankstoemb edding, it’spossibleand som etimesusefulfor unname d st ruc t typestohave

methodstoo.

Here’sanice trick to illustrate. Thisexampleshows par t of a simplecache imp lemente d using

twopackage-le vel var iables, a mut ex(§9.2) andthe map thatitguard s:

var (

mu sync.Mutex // guards mapping

mapping = make(map[string]string)

)

func Lookup(key string) string {

mu.Lock()

v:=mapping[key]

mu.Unlock()

return v

}

Theversionbelow isfunctionallyequivalentbut gro ups toget her the two rel ate d var iables in a

singlepackage-le vel var iable, cache:

var cache = struct {

sync.Mutex

mapping map[string]string

}{

mapping: make(map[string]string),

}

func Lookup(key string) string {

cache.Lock()

v:=cache.mapping[key]

cache.Unlock()

return v

}

Thenew var iable gives moreexpressivenames tothe var iables rel ate d to the cache,and

becaus e the sync.Mutex ﬁeldisemb edde d within it, its Lock and Unlock methodsare

prom ote d to the unnamed str uct typ e,allow ing ustolockthe cache with a self-explanatory

sy ntax.

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

164 CHAPTER 6. METHODS

6.4. Method Values and Expressions

Us ual lyweselec t andcal l amet hod inthe sameexpression, as in p.Distance(),but it’spossi-

bletosep aratethese two operat ions.The selec tor p.Distance yields a metho d valu e,afunc-

tion thatbinds a met hod (Point.Distance)toaspeciﬁcreceivervalue p.Thisfunctioncan

then beinv oke d withoutareceivervalue; itneedsonlythe non-receiverarguments.

p:=Point{1, 2}

q:=Point{4, 6}

distanceFromP := p.Distance // method value

fmt.Println(distanceFromP(q)) // "5"

var origin Point // {0, 0}

fmt.Println(distanceFromP(origin)) // "2.23606797749979", ;5

scaleP := p.ScaleBy // method value

scaleP(2) // pbecomes (2, 4)

scaleP(3) // then (6, 12)

scaleP(10) // then (60, 120)

Method values areusefulwhenapackage’sAPI cal lsfor a functionvalue,and the client’s

desired beh avior for thatfunctionistocal l amet hod onaspeciﬁcreceiver. For example, the

func tion time.AfterFunc callsafunctionvalue afteraspeciﬁed del ay. Thisprogram usesit

to launch the rocket r af ter 10 secon ds:

type Rocket struct { /* ... */ }

func (r *Rocket) Launch() { /* ... */ }

r:=new(Rocket)

time.AfterFunc(10 * time.Second, func() { r.Launch() })

Themet hod value synt axisshorter :

time.AfterFunc(10 * time.Second, r.Launch)

Relate d to the method value isthe metho d expre ssi on.Whencal lingamet hod,asopp osedto

an ordinar y func tion, wemustsup ply the receiverinaspeci al way usingthe selec tor syntax. A

method expression, writt en T.f or (*T).f where T is a typ e,yieldsafunctionvalue wit h areg-

ular ﬁrs t parameter tak ingthe place ofthe receiver, soitcan becal le d in the usu alway.

p:=Point{1, 2}

q:=Point{4, 6}

distance := Point.Distance // method expression

fmt.Println(distance(p, q)) // "5"

fmt.Printf("%T\n", distance) // "func(Point, Point) float64"

scale := (*Point).ScaleBy

scale(&p, 2)

fmt.Println(p) // "{2 4}"

fmt.Printf("%T\n", scale) // "func(*Point, float64)"

Method expressions can behelpf ulwhenyou need a value torepresent a choice amon g several

methodsbelon gingtothe sametyp e so thatyou can cal l thechosenmet hod wit h many

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

SECTION 6.5. EXAMPLE: BIT VECTOR TYPE 165

dif ferentreceivers.Inthe fol low ing example, the var iable op repres ents either the addition or

thesubtrac tionmet hod oftyp e Point,and Path.TranslateBy callsitfor eachpoint inthe

Path:

type Point struct{ X, Y float64 }

func (p Point) Add(q Point) Point { return Point{p.X + q.X, p.Y + q.Y} }

func (p Point) Sub(q Point) Point { return Point{p.X - q.X, p.Y - q.Y} }

type Path []Point

func (path Path) TranslateBy(offset Point, add bool) {

var op func(p, q Point) Point

if add {

op = Point.Add

}else {

op = Point.Sub

}

for i := range path {

// Call either path[i].Add(offset) or path[i].Sub(offset).

path[i] = op(path[i], offset)

}

6.5. Example: BitVector Type

Sets in Goare usu allyimp lemente d as a map[T]bool,where T is the elementtyp e.Asetrep-

resent edbyamap isver y ﬂexiblebut,for cer tain pro blems, a speci alizedrepresent ation may

outp erfor m it.For example, indom ainssuchasdat aﬂow analysiswhere set elements aresmall

non-negat ive int egers,sets havemanyelements, andset operat ions likeunionand int ers ection

arecommon, a bitvec tor is ide al.

Abit vec tor usesaslice ofunsig ned int egervalues or ‘‘word s,’’ each bit of whichrepresentsa

possible elementofthe set. Theset cont ains i if the i-t h bit isset. Thefol low ing program

demon strates a simplebit vec tor typ e with thre e methods:

gopl.io/ch6/intset

// An IntSet is a set of small non-negative integers.

// Its zero value represents the empty set.

type IntSet struct {

words []uint64

}

// Has reports whether the set contains the non-negative value x.

func (s *IntSet) Has(x int) bool {

word, bit := x/64, uint(x%64)

return word < len(s.words) && s.words[word]&(1<<bit) != 0

}

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

166 CHAPTER 6. METHODS

// Add adds the non-negative value x to the set.

func (s *IntSet) Add(x int) {

word, bit := x/64, uint(x%64)

for word >= len(s.words) {

s.words = append(s.words, 0)

}

s.words[word] |= 1 << bit

}

// UnionWith sets s to the union of s and t.

func (s *IntSet) UnionWith(t *IntSet) {

for i, tword := range t.words {

if i < len(s.words) {

s.words[i] |= tword

}else {

s.words = append(s.words, tword)

}

Sinceeachwordhas 64 bits, tolocatethe bit for x,weuse the quotient x/64 as the wordindex

andthe remainder x%64 as the bit index wit hin thatword. The UnionWith op erat ionusesthe

bit w ise ORoperator | to compute the union64elements at a time. (We’ll revisit the choice of

64-bit words inExercis e 6.5.)

Thisimp lementation lacks manydesirable features, som e of whichare pos edasexercis es

below, but oneishardtolivewit hout: way toprint an IntSet as a str ing . Let’sgiveitaString

method aswedid wit h Celsius in Sec tion 2.5:

// String returns the set as a string of the form "{1 2 3}".

func (s *IntSet) String() string {

var buf bytes.Buffer

buf.WriteByte('{')

for i, word := range s.words {

if word == 0 {

continue

}

for j := 0; j < 64; j++ {

if word&(1<<uint(j)) != 0 {

if buf.Len() > len("{") {

buf.WriteByte('')

}

fmt.Fprintf(&buf, "%d", 64*i+j)

}

buf.WriteByte('}')

return buf.String()

}

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

SECTION 6.5. EXAMPLE: BIT VECTOR TYPE 167

No tice the simi lar ity ofthe String method above wit h intsToString in Sec tion 3.5.4;

bytes.Buffer is often usedthisway in String methods. The fmt packagetre ats typ es with a

String method speci ally sothatvalues ofcomplic ated typ es candispl aythems elves in a user-

friendlymanner.Ins teadofprint ing the raw represent ation of the value (a str uct inthiscas e),

fmt callsthe String method.The mech anism relies onint erfaces andtyp e assertions,which

we’ll explain in Chapt er7.

We can now demon strate IntSet in action:

var x, y IntSet

x.Add(1)

x.Add(144)

x.Add(9)

fmt.Println(x.String()) // "{1 9 144}"

y.Add(9)

y.Add(42)

fmt.Println(y.String()) // "{9 42}"

x.UnionWith(&y)

fmt.Println(x.String()) // "{1 9 42 144}"

fmt.Println(x.Has(9), x.Has(123)) // "true false"

Awordofcaution:wedeclare d String and Has as met hodsofthe point ertyp e *IntSet not

outofnecessity,but for con sistenc y with theother two met hods, whichneed a point erreceiver

becaus e they assig n to s.words.Con sequently, an IntSet valu e do es nothaveaString

method,occ asionallyleadingtosur priseslikethis:

fmt.Println(&x) // "{1 9 42 144}"

fmt.Println(x.String()) // "{1 9 42 144}"

fmt.Println(x) // "{[4398046511618 0 65536]}"

In the ﬁrs t case,weprint an *IntSet pointer, whichdoeshaveaString method.Inthe

second cas e,wecal l String() on an IntSet var iable; the compi ler insertsthe imp licit & op er-

at ion, giv ing usapoint er, whichhas the String method.But inthe thirdcas e,because the

IntSet value does not haveaString method, fmt.Println pr intsthe represent ation of the

st ruc t instead. It’simp ortantnot tofor g etthe & op erator.Mak ing String amet hod of

IntSet,not *IntSet,mig htbeagood ide a, butthisisacas e-by-c asejudg ment.

Exercis e 6.1: Implementthese addition almet hods:

func (*IntSet) Len() int // return the number of elements

func (*IntSet) Remove(x int) // remove x from the set

func (*IntSet) Clear() // remove all elements from the set

func (*IntSet) Copy() *IntSet // return a copy of the set

Exercis e 6.2: Deﬁne a var iadic (*IntSet).AddAll(...int) method thatallowsalistofval-

ues tobeadde d,suchas s.AddAll(1, 2, 3).

Exercis e 6.3: (*IntSet).UnionWith comp utesthe unionoftwo sets using |,the word-p aral-

lelbit w ise ORoperator.Imp lementmet hodsfor IntersectWith, DifferenceWith,and Sym-

metricDifference forthe cor respondingset operat ions.(Thesymmetr icdif ference oftwo

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

168 CHAPTER 6. METHODS

sets cont ainsthe elements present inone set orthe other but not bot h.)

Exercis e 6.4: Addamet hod Elems that retur nsaslice cont ainingthe elements ofthe set, suit-

able for iterat ingoverwit h a range lo op.

Exercis e 6.5: Thetyp e of eachwordusedby IntSet is uint64,but 64-bit arithmeticmay be

inefﬁcientona32-bit platfor m.Modif y theprogram touse the uint type,whichisthe most

efﬁcientunsig ned int egertyp e forthe platfor m.Ins teadofdiv idingby64, deﬁne a con stant

holdingthe effec tive size of uint in bits, 32 or64. You can use the perhaps too-cle ver expres-

sion 32 << (^uint(0) >> 63) forthispur pos e.

6.6. Encapsulation

Avar iable ormet hod ofanobj e ctissaid tobe encapsul ate d if itisinaccessibletoclients ofthe

objec t.Enc apsulat ion, som etimescal le d in f o rmati onhiding,isakey asp ect ofobj e ct-oriente d

prog ramming.

Go has onlyone mech anism tocontrol the visibilit y of names: capit alize d identiﬁersare

exp orted fro m thepackageinwhichthe y aredeﬁne d,and unc apit alize d namesare not.The

same mech anism thatlimits accesstomembers ofapackageals o limits accesstothe ﬁelds ofa

st ruc t or the methodsofatyp e.Asacon sequence,toenc apsulateanobj e ct, wemustmakeita

st ruc t.

That’sthe reasonthe IntSet type fro m thepre vious sec tion was decl are d as a str uct typ e even

though ithas onlyasingleﬁeld:

type IntSet struct {

words []uint64

}

We cou ldins teaddeﬁne IntSet as a slice typ e as fol lows, thoug h of cours e we’dhavetoreplace

each occ urrenceof s.words by *s in its met hods:

type IntSet []uint64

Although thisversionof IntSet wouldbeess ent ial lyequivalent, itwou ldallow clients fro m

ot her packages toreadand modif y theslice direc tly.Put another way,where asthe expression

*s couldbeusedinany package, s.words mayapp ear onlyinthe packagethatdeﬁnes

IntSet.

Anot her cons equence ofthisname-b ased mechanism isthatthe unitofenc apsulat ionisthe

package, not the typ e as in manyother langu ages. The ﬁelds ofastr uct typ e arevisible toall

co de within the samepackage. Whether the codeapp earsinafunctionoramet hod makes no

dif ference.

Encapsulat ionprovides three beneﬁts. First,because clients cannot direc tly modif y the

objec t’svar iables, one need ins pec t fe wer statementstounderst and the possiblevalues ofthose

var iables.

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

SECTION 6.6. ENCAPSULATION 169

Second,hidingimp lementation det ails pre vents clients fro m dep endingonthings thatmig ht

ch ange , whichgives the desig ner gre aterfre e dom toevo l ve theimp lementation wit houtbre ak-

ingAPI compatibi lit y.

As anexample, con sider the bytes.Buffer type.Itisfre quentlyusedtoacc umulatever y

short str ings, soitisaproﬁtableopt imizat iontores erve a lit tle extra space in the obj e ctto

avoidmemor y al location inthiscommoncas e.Since Buffer is a str uct typ e,thisspace takes

thefor m of anext ra ﬁeldoftyp e [64]byte with anunc apit alize d name.Whenthis ﬁeldwas

adde d,because itwas not exp orted,clients of Buffer outside the bytes packagewereunaware

of any change exceptimp rov edper for mance. Buffer andits Grow method are shown below,

simpliﬁed for clarity :

type Buffer struct {

buf []byte

initial [64]byte

/* ... */

}

// Grow expands the buffer'scapacity, if necessary,

// to guarantee space for another n bytes. [...]

func (b *Buffer) Grow(n int) {

if b.buf == nil {

b.buf = b.initial[:0] // use preallocated space initially

}

if len(b.buf)+n > cap(b.buf) {

buf := make([]byte, b.Len(), 2*cap(b.buf) + n)

copy(buf, b.buf)

b.buf = buf

}

Thethirdbeneﬁtofenc apsulat ion, andinmanycas es themostimp ortant, isthatitpre vents

clients fro m sett ing anobj e ct’svar iables arbit rar ily.Because the obj e ct’svar iables can beset

on lybyfunctions inthe samepackage, the aut hor of thatpackagecan ensure thatall those

func tions maintain the obj e ct’sint ernal invar iants. For example, the Counter type belowper-

mits clients toincrement the count erortores et it tozero, but not toset ittosom e arbit rar y

value:

type Counter struct { n int }

func (c *Counter) N() int {return c.n }

func (c *Counter) Increment() { c.n++ }

func (c *Counter) Reset() {c.n = 0 }

Func tions thatmerelyaccessormodif y internal values ofatyp e,suchasthe methodsofthe

Logger type fro m log package, below,are cal le d getters and setters.How ever, whennaminga

gett ermet hod,weusu allyomitthe Get preﬁx. Thispreferencefor brevit y extends toall met h-

ods, not justﬁeldaccessors,and toother redundant preﬁxesaswel l,suchas Fetch, Find,and

Lookup.

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

170 CHAPTER 6. METHODS

package log

type Logger struct {

flags int

prefix string

// ...

}

func (l *Logger) Flags() int

func (l *Logger) SetFlags(flag int)

func (l *Logger) Prefix() string

func (l *Logger) SetPrefix(prefix string)

Go sty ledoesnot forbidexp orted ﬁelds.Ofcours e, on ceexp orted,aﬁeldcannot beunex-

ported wit houtanincompatiblechange tothe API,sothe initial choice shouldbedelib erate

andshouldcon sider the complexity ofthe invar iants thatmustbemaintained,the likelihood

of fut ure changes, andthe quant ity ofclientcodethatwou ldbeaffec ted byachange .

Encapsulat ionisnot always desirable.Byrevealingits represent ation asan int64 numb erof

nanosecond s, time.Duration lets ususe all the usu alarithmeticand comparisonoperat ions

with durat ions,and eventodeﬁne con stantsofthistyp e:

const day = 24 * time.Hour

fmt.Println(day.Seconds()) // "86400"

As another example, contrast IntSet with the geometry.Path type fro m thebeg inningofthis

ch apt er. Path was deﬁne d as a slice typ e,allow ing its clients tocon str uct ins tances usingthe

slice literal synt ax, toiterateoverits pointsusingarange loop, and soon, where asthese opera-

tion s aredenie d to clients of IntSet.

Here’sthe cruci al dif ference: geometry.Path is int rinsic ally a sequence ofpoints, nomoreand

no less, andwedon’tfores eeaddingnew ﬁelds toit, soitmakes sense for the geometry pack-

agetorevealthat Path is a slice.Incontrast, an IntSet mere lyhappens toberepresent edasa

[]uint64 slice.Itcou ldhavebeenrepresent edusing []uint,orsom ethingcompletelydif fer-

entfor sets thatare spars e or ver y smal l,and itmig htperhaps beneﬁtfro m addition alfeatures

li ke an ext ra ﬁeldtorecordthe numberofelements in the set. For these reasons,itmakes

sens e for IntSet to beopaque.

In thischapt er, welearned how toass oci atemet hodswit h name d types, andhow tocal l thos e

methods. Alt hough met hodsare cruci al to obj e ct-oriente d prog ramming, the y’re onlyhalf the

picture. Tocompleteit, weneed interfaces,the subjec t of the next chapt er.

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

Int erfaces

Interface typ es express generalizat ions orabstrac tions about the beh avior s of other typ es. By

generalizing, int erfaces let uswrite functions thatare more ﬂexibleand adapt ablebecause the y

arenot tie d to the det ails ofone par tic ularimp lementation.

Many obj e ct-oriente d languageshavesom e notion of int erfaces, but whatmakes Go’sint er-

faces sodistinc tiveisthatthe y are sati sﬁed imp licit ly.Inother words,there’snoneed to

de clare all the int erfaces thatagiven concrete typ e satisﬁes; simply possessingthe necessary

methodsisenoug h. Thisdesig n lets you cre ate new int erfaces thatare sat isﬁed byexist ing

concrete typ es withoutchang ing the exist ing typ es, whichispar tic ularlyusefulfor typ es

deﬁne d in packages thatyou don’tcontrol .

In thischapt er, we’ll start bylooking atthe basic mechanics ofint erface typ es andtheir values.

Alon g theway,we’ll study several importantint erfaces fro m thestand ard librar y.ManyGo

prog ramsmakeasmuchuse ofstand ard int erfaces as the y do oftheir own ones. Final ly, we’ll

lo okat ty peass ertions (§7.10) and ty peswitches (§7.13) andsee how the y enable a different

kind ofgenerality.

7.1. Int erfaces as Contracts

Al l thetyp es we’ve looke d at sofar havebeen concre tetypes.Aconcrete typ e sp eciﬁes the

exac t repres entation of itsvalues andexp oses the int rinsic operat ions ofthatrepresent ation,

such asarithmeticfor numbers,orindexing, append,and range forslices. A concrete typ e

mayals o prov ide addition albeh avior s thro ugh its met hods. Whenyou haveavalue ofacon-

cretetyp e,you knowexac tly whatit is andwhatyou can do with it.

Thereisanother kindoftyp e in Gocal le d an interface type.Anint erface isan ab strac t ty pe.It

do esn’texp osethe represent ation or int ernal str uctureofits values, orthe set ofbasic

171

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

172 CHAPTER 7. INTERFACES

op erat ions the y supp ort;itreveals onlysom e of their met hods. Whenyou haveavalue ofan

interface typ e,you knownot hingabout whatit is;you knowonlywhatitcan do,ormore

precis ely,whatbeh avior s areprovide d by its met hods.

Throughoutthe book, we’ve beenusingtwo simi lar functions for str ing for matting:

fmt.Printf,whichwritesthe resulttothe stand ard out put (aﬁle), and fmt.Sprintf,which

returnsthe resultasastring.Itwou ldbeunfor tunateifthe hardpar t, formatting the result,

hadtobedup lic ated because ofthese sup erﬁci al dif ferences in how the resultisused. Thanks

to int erfaces, itdoesnot.Bot h of these functions are , in effe ct, wrappers aro und a thirdfunc-

tion, fmt.Fprintf,thatisagnosticabout whathappens tothe resultitcomputes:

package fmt

func Fprintf(w io.Writer, format string, args ...interface{}) (int, error)

func Printf(format string, args ...interface{}) (int, error) {

return Fprintf(os.Stdout, format, args...)

}

func Sprintf(format string, args ...interface{}) string {

var buf bytes.Buffer

Fprintf(&buf, format, args...)

return buf.String()

}

The F preﬁxof Fprintf st and s for ﬁle andindic ates thatthe for matte d output shouldbe

wr itt entothe ﬁle provide d as the ﬁrs t argument. Inthe Printf case,the argument, os.Std-

out,isan *os.File.Inthe Sprintf case,how ever, the argumentisnot a ﬁle,thoug h it sup er-

ﬁci ally res embles one: &buf is a point ertoamemor y buffer towhichbytes can bewritt en.

The ﬁrs t parameter of Fprintf is not a ﬁle either.It’san io.Writer,whichisanint erface typ e

with thefol low ing declarat ion:

package io

// Writer is the interface that wraps the basic Write method.

type Writer interface {

// Write writes len(p) bytes from p to the underlying data stream.

// It returns the number of bytes written from p (0 <= n <= len(p))

// and any error encountered that caused the write to stop early.

// Write must return a non-nil error if it returns n < len(p).

// Write must not modify the slice data, even temporarily.

// Implementations must not retain p.

Write(p []byte) (n int, err error)

}

The io.Writer interface deﬁnesthe contrac t between Fprintf andits cal lers. On theone

hand,the contrac t re quires thatthe cal ler prov ide a value ofacon crete typ e li ke *os.File or

*bytes.Buffer that has a met hod cal le d Write with theappro priatesig natureand beh avior.

On the other hand, the contrac t gu arante esthat Fprintf wi l l do its job given anyvalue that

satisﬁes the io.Writer interface. Fprintf maynot assume thatitiswriting toaﬁle orto

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

SECTION 7.1. INTERFACES AS CONTRACTS173

memory,onlythatitcan cal l Write.

Becaus e fmt.Fprintf assumesnot hingabout the represent ation of the value andrelies only

on the beh avior s gu arante e d by the io.Writer cont rac t, we can safelypassavalue ofany con-

cretetyp e that sat isﬁes io.Writer as the ﬁrs t argumentto fmt.Fprintf.Thisfre e dom to

su bst itute one typ e foranother thatsat isﬁes the sameint erface iscal le d substitutabi lity,and is

ahal lmarkofobj e ct-oriente d prog ramming.

Let’stestthisout usinganew typ e.The Write method ofthe *ByteCounter type below

mere lycountsthe bytes writt entoitbeforedis cardingthem. (Theconversionisrequired to

make the typ es of len(p) and *c match inthe += assig nmentstatement.)

gopl.io/ch7/bytecounter

type ByteCounter int

func (c *ByteCounter) Write(p []byte) (int, error) {

*c += ByteCounter(len(p)) // convert int to ByteCounter

return len(p), nil

}

Since *ByteCounter satisﬁes the io.Writer cont rac t, we can passitto Fprintf,whichdoes

itsstr ing for matting obliv iou s to thischange;the ByteCounter correc tly acc umulates the

lengt h of the result.

var c ByteCounter

c.Write([]byte("hello"))

fmt.Println(c) // "5", = len("hello")

c=0//reset the counter

var name = "Dolly"

fmt.Fprintf(&c, "hello, %s", name)

fmt.Println(c) // "12", = len("hello, Dolly")

Besides io.Writer,there isanother interface ofgre atimp ortance tothe fmt package.

Fprintf and Fprintln prov ide a way for typ es to control how their values areprint ed. In

Section2.5, wedeﬁne d a String method for the Celsius type sothattemperatures wou ld

pr int as "100°C",and inSec tion 6.5 weequip ped *IntSet with a String method sothatsets

wouldberendered usingtradition alset not ation like "{1 2 3}".Declaring a String method

makesatyp e satisf y on e of the most widely usedint erfaces ofall, fmt.Stringer:

package fmt

// The String method is used to print values passed

// as an operand to any format that accepts a string

// or to an unformatted printer such as Print.

type Stringer interface {

String() string

}

We’ll explain how the fmt packagedis cov ers whichvalues sat isf y this int erface in Sec tion 7.10.

Exercis e 7.1: Usingthe ide as from ByteCounter,imp lementcount ers for words and for lines.

Yo u wi l l ﬁnd bufio.ScanWords us eful.

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

174 CHAPTER 7. INTERFACES

Exercis e 7.2: Wr ite a function CountingWriter with thesig naturebelow that, given an

io.Writer,retur nsanew Writer that wraps the original, and a point ertoan int64 var iable

that at anymom ent containsthe numberofbytes writt entothe new Writer.

func CountingWriter(w io.Writer) (io.Writer, *int64)

Exercis e 7.3: Wr ite a String method for the *tree type in gopl.io/ch4/treesort (§4.4)

that reveals the sequence ofvalues in the tre e.

7.2. Int erfaceTypes

An int erface typ e sp eciﬁes a set ofmet hodsthatacon crete typ e mu stpossess tobecon sidered

an inst anceofthatint erface.

The io.Writer type isone ofthe most widely usedint erfaces because itprovides an abstrac-

tion of all the typ es to whichbytes can bewritt en, whichincludes ﬁles, memor y buffers, net-

work conne ction s,HTTPclients, archivers, hashers,and soon. The io packagedeﬁnesmany

ot her usefulint erfaces. A Reader repres ents anytyp e from whichyou can readbytes, anda

Closer is any value thatyou can clos e,suchasaﬁle oranet wor k connec tion.(By now you’ve

prob ably not ice d thenamingconvent ion for manyofGo’ssingle-met hod int erfaces.)

package io

type Reader interface {

Read(p []byte) (n int, err error)

}

type Closer interface {

Close() error

}

Looking far ther, weﬁnd declarat ions ofnew int erface typ es as combinat ions ofexist ing ones.

Here are two examples:

type ReadWriter interface {

Reader

Writer

}

type ReadWriteCloser interface {

Reader

Writer

Closer

}

Thesyntaxusedabove , whichres embles str uct emb edding, lets usnameanother interface as a

shorthand for writing out all ofits met hods. Thisiscal le d embedding an interface.Wecou ld

have writt en io.ReadWriter withoutemb edding, alb eit lesssuccinctly,likethis:

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

SECTION 7.3. INTERFACE SATISFACTION 175

type ReadWriter interface {

Read(p []byte) (n int, err error)

Write(p []byte) (n int, err error)

}

or evenusingamixtureofthe two sty les:

type ReadWriter interface {

Read(p []byte) (n int, err error)

Writer

}

Al l thre e de clarat ions havethe sameeffec t.The order in whichthe methodsapp ear isimma-

terial.All thatmatters isthe set ofmet hods.

Exercis e 7.4: The strings.NewReader func tionretur nsavalue thatsat isﬁes the io.Reader

interface (andothers) byreadingfro m itsargument, a str ing . Implementasimpleversionof

NewReader yourself,and use ittomakethe HTMLparser(§5.2) takeinp utfro m astr ing .

Exercis e 7.5: The LimitReader func tioninthe io packageaccepts an io.Reader r anda

numb erofbytes n,and retur nsanother Reader that reads fro m r butrep orts an end-of-ﬁle

condit ion after n bytes. Imp lementit.

func LimitReader(r io.Reader, n int64) io.Reader

7.3. Int erfaceSatisfaction

Atyp e sati sﬁes an interface if itpossess esall the methodsthe int erface requires. For example,

an *os.File satisﬁes io.Reader, Writer, Closer,and ReadWriter.A*bytes.Buffer sat-

isﬁes Reader, Writer,and ReadWriter,but doesnot sat isf y Closer becaus e it doesnot havea

Close method.Asashorthand,Goprogrammersoften say thatacon crete typ e ‘‘is a’’ partic u-

larint erface typ e,meaningthatitsat isﬁes the int erface.For example, a *bytes.Buffer is an

io.Writer;an *os.File is an io.ReadWriter.

Theassig nabilit y rule (§2.4.2) for int erfaces isver y simple: an expressionmay beassig ned to

an interface only if its typ e satisﬁes the int erface.So:

var w io.Writer

w=os.Stdout // OK: *os.File has Write method

w=new(bytes.Buffer) // OK: *bytes.Buffer has Write method

w=time.Second // compile error: time.Duration lacks Write method

var rwc io.ReadWriteCloser

rwc = os.Stdout // OK: *os.File has Read, Write, Close methods

rwc = new(bytes.Buffer) // compile error: *bytes.Buffer lacks Close method

Thisruleapp lies evenwhenthe rig ht-handside isits elf an interface:

w=rwc // OK: io.ReadWriteCloser has Write method

rwc = w // compile error: io.Writer lacks Close method

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

176 CHAPTER 7. INTERFACES

Becaus e ReadWriter and ReadWriteCloser include allthe methodsof Writer,any typ e that

satisﬁes ReadWriter or ReadWriteCloser ne cessarily sat isﬁes Writer.

Before wegofur ther, weshouldexplain one subtlet y in whatitmeans for a typ e to havea

method.Recal l from Sec tion 6.2 thatfor eachnamed con crete typ e T,som e of itsmet hods

have a receiveroftyp e T itself where asothersrequirea*T pointer. Recal l also thatitislegal to

call a *T method onanargumentoftyp e T so lon g as the argumentisavariab le;the compi ler

implicitlytakes its address. But thisismeresyntactic sugar : avalue oftyp e T do es notpossess

al l themet hodsthata*T pointerdoes, and as a resultitmig htsat isf y fe wer interfaces.

An examplewill makethiscle ar.The String method ofthe IntSet type fro m Section6.5

re quires a point erreceiver, sowecannot cal l that met hod onanon-addressable IntSet value:

type IntSet struct { /* ... */ }

func (*IntSet) String() string

var _ = IntSet{}.String() // compile error: String requires *IntSet receiver

butwecan cal l it on an IntSet var iable:

var s IntSet

var _ = s.String() // OK: s is a variable and &s has a String method

Ho wever,since only *IntSet hasaString method,only *IntSet satisﬁes the fmt.Stringer

interface:

var _ fmt.Stringer = &s // OK

var _ fmt.Stringer = s // compile error: IntSet lacks String method

Section12.8 includes a program thatprintsthe methodsofanarbit rar y value,and the

godoc -analysis=type to ol(§10.7.4) displays the methodsofeachtyp e andthe rel ation ship

betweenint erfaces andcon crete typ es.

Like anenv elope thatwraps andcon ceals the letteritholds,anint erface wraps andcon ceals

thecon crete typ e andvalue thatitholds.Onlythe methodsreveale d by the int erface typ e may

be cal le d,even if the con crete typ e hasothers:

os.Stdout.Write([]byte("hello")) // OK: *os.File has Write method

os.Stdout.Close() // OK: *os.File has Close method

var w io.Writer

w=os.Stdout

w.Write([]byte("hello")) // OK: io.Writer has Write method

w.Close() // compile error: io.Writer lacks Close method

An int erface wit h more met hods, suchas io.ReadWriter,tel lsusmoreabout the values it

cont ains, andplaces gre aterdemands onthe typ es that implementit, thandoesanint erface

with fewer met hodssuchas io.Reader.Sowhatdoesthe typ e interface{},whichhas no

methods at all,tel l us about the con crete typ es that sat isf y it?

That’srig ht: not hing. Thismay seemuseless, but infac t thetyp e interface{},whichis

called the emptyint erface type,isindispens able. Because the emp tyint erface typ e pl aces no

demands onthe typ es that sat isf y it,wecan assig n any value tothe emp tyint erface.

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

SECTION 7.3. INTERFACE SATISFACTION 177

var any interface{}

any = true

any = 12.34

any = "hello"

any = map[string]int{"one": 1}

any = new(bytes.Buffer)

Although itwasn’tobv iou s,we’ve beenusingthe emp tyint erface typ e since the ver y ﬁrs t

exampleinthisbook, because itiswhatallowsfunctions like fmt.Println,or errorf in

Section5.7, toacceptarguments ofany typ e.

Of course,hav ingcre ate d an interface{} value cont ainingaboole an, ﬂoat, str ing , map,

pointer, orany other typ e,wecan do not hingdirec tly tothe value itholds since the int erface

hasnomet hods. Weneed a way toget the value backout again. We’ll see how todothat

usingaty peass ertion in Sec tion 7.10.

Sinceint erface sat isfac tiondep ends onlyonthe methodsofthe two typ es invo l ved, there isno

ne e d to declare the rel ation shipbet weena con crete typ e andthe int erfaces itsat isﬁes. That

said,itisocc asionallyusefultodocumentand ass ert the rel ation shipwhenitisint ended but

notother wis e enforce d by the program. Thedeclarat ionbelow ass ertsatcompi letimethata

value oftyp e *bytes.Buffer satisﬁes io.Writer:

// *bytes.Buffer must satisfy io.Writer

var w io.Writer = new(bytes.Buffer)

We needn’tallocateanew var iable since anyvalue oftyp e *bytes.Buffer wi l l do,even nil,

whichwewrite as (*bytes.Buffer)(nil) usinganexplicitconversion. And since wenever

intend torefer to w,wecan replace itwit h theblank identiﬁer.Toget her,these changesgiveus

this morefrugal variant:

// *bytes.Buffer must satisfy io.Writer

var _ io.Writer = (*bytes.Buffer)(nil)

No n-empt y interface typ es such as io.Writer aremostoften sat isﬁed byapoint ertyp e,par-

ticularlywhenone ormoreofthe int erface met hodsimp lies som e kind ofmut ation tothe

re ceiver, asthe Write method does. A pointertoastr uct isanesp eci ally commonmet hod-

bear ingtyp e.

Butpoint ertyp es arebynomeans the onlytyp es that sat isf y interfaces, andevenint erfaces

with mut atormet hodsmay besat isﬁed byone ofGo’sother reference typ es. We’ve seenexam-

ples ofslice typ es with met hods(geometry.Path,§6.1) andmap typ es with met hods

(url.Values,§6.2.1), andlater we’ ll see a functiontyp e with met hods(http.HandlerFunc,

§7.7). Evenbasic typ es maysat isf y interfaces; as wesaw in Sec tion 7.4, time.Duration sat-

isﬁes fmt.Stringer.

Acon crete typ e maysat isf y many unrel ate d interfaces. Con sider a program thatorganizes or

sellsdig itize d cult ural artifac ts li kemusic, ﬁlms, andbooks. Itmig htdeﬁne the fol low ing set

of con crete typ es:

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

178 CHAPTER 7. INTERFACES

Album

Book

Movie

Magazine

Podcast

TVEpisode

Track

We can express eachabstrac tionofint erest asanint erface.Som e prop erties arecommontoall

ar tifac ts, such asatit le, a cre ation date, and a listofcre ators (author s or artists).

type Artifact interface {

Title() string

Creators() []string

Created() time.Time

}

Ot her pro per ties arerestr icted tocer tain typ es of artifac ts. Prop erties ofthe print edwordare

re levantonlytobooks andmagazines, where asonlymov ies andTVepi sodes haveascreen

resolut ion.

type Text interface {

Pages() int

Words() int

PageSize() int

}

type Audio interface {

Stream() (io.ReadCloser, error)

RunningTime() time.Duration

Format() string // e.g., "MP3", "WAV"

}

type Video interface {

Stream() (io.ReadCloser, error)

RunningTime() time.Duration

Format() string // e.g., "MP4", "WMV"

Resolution() (x, y int)

}

Thes e interfaces arebut oneusefulway togro uprel ate d concrete typ es together andexpress

thefacets the y sh are incommon. Wemay discov erother gro upingslater.For example, ifwe

ﬁnd weneed tohandle Audio and Video it ems inthe sameway,wecan deﬁne a Streamer

interface torepresent their common asp ectswit houtchang ing any exist ing typ e de clarat ions.

type Streamer interface {

Stream() (io.ReadCloser, error)

RunningTime() time.Duration

Format() string

}

Each gro uping ofcon crete typ es basedontheir share d behavior s canbeexpress edasanint er-

face typ e.Unlikeclass-b ased langu ages, in whichthe set ofint erfaces sat isﬁed byaclass is

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

SECTION 7.4. PARSING FLAGS WITH FLAG.VALUE179

explicit, in Gowecan deﬁne newabstrac tions orgro upingsofint erest whenweneed them,

withoutmodif yingthe declarat ionofthe con crete typ e.Thisispar tic ularlyusefulwhenthe

concrete typ e comesfro m apackagewritt enbyadif ferentaut hor.Ofcours e, theredoneed to

be underly ing commonalities in the con crete typ es.

7.4. Parsing Flags with flag.Value

In thissec tion,we’ll see how another stand ard int erface, flag.Value,helps usdeﬁne new

notation s forcommand-line ﬂags. Con sider the program below,whichsle eps foraspeciﬁed

period oftime.

gopl.io/ch7/sleep

var period = flag.Duration("period", 1*time.Second, "sleep period")

func main() {

flag.Parse()

fmt.Printf("Sleeping for %v...", *period)

time.Sleep(*period)

fmt.Println()

}

Before itgoestosle ep it prints the timeper iod. The fmt packagecal lsthe time.Duration’s

String method toprint the per iodnot asanumberofnanos econd s,but inauser-friendly

notation:

$gobuild gopl.io/ch7/sleep

$./sleep

Sleeping for 1s...

By defau lt, the sleepper iodisone secon d,but it can becontrol led through the -period com-

mand-line ﬂag . The flag.Duration func tioncre atesaﬂag variableoftyp e time.Duration

andallowsthe usertospecif y thedurat ioninavar ietyofuser-friendlyfor mats, includingthe

same notation printe d by the String method.Thissymmetr y of desig n le adstoanice user

interface.

$./sleep -period 50ms

Sleeping for 50ms...

$./sleep -period 2m30s

Sleeping for 2m30s...

$./sleep -period 1.5h

Sleeping for 1h30m0s...

$./sleep -period "1 day"

invalid value "1 day" for flag -period: time: invalid duration 1 day

Becaus e durat ion-value d ﬂags aresouseful, thisfeature isbui ltint o the flag package, but it’s

easy todeﬁne newﬂag not ation s forour own data typ es. We need onlydeﬁne a typ e that sat-

isﬁes the flag.Value interface,whose declarat ionisbelow :

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

180 CHAPTER 7. INTERFACES

package flag

// Value is the interface to the value stored in a flag.

type Value interface {

String() string

Set(string) error

}

The String method for mats the ﬂag’svalue for use incommand-linehelpmessages; thu s

ever y flag.Value is als o a fmt.Stringer.The Set method parsesits str ing argumentand

up dates the ﬂag value.Ineffec t,the Set method isthe inv ers e of the String method,and itis

go o d prac tice for themtouse the samenot ation.

Let’sdeﬁne a celsiusFlag type thatallowsatemperaturetobespeciﬁed inCel siu s,orin

Fahren heitwit h an appropriateconversion. Not ice that celsiusFlag embeds a Celsius

(§2.5), there bygetting a String method for fre e.Tosat isf y flag.Value,weneed onlydeclare

the Set method:

gopl.io/ch7/tempconv

// *celsiusFlag satisfies the flag.Value interface.

type celsiusFlag struct{ Celsius }

func (f *celsiusFlag) Set(s string) error {

var unit string

var value float64

fmt.Sscanf(s, "%f%s", &value, &unit) // no error check needed

switch unit {

case "C", "°C":

f.Celsius = Celsius(value)

return nil

case "F", "°F":

f.Celsius = FToC(Fahrenheit(value))

return nil

}

return fmt.Errorf("invalid temperature %q", s)

}

Thecal l to fmt.Sscanf pars esaﬂoating-p ointnumber(value)and a str ing (unit)fro m the

input s.Alt hough one mustusu allyche ck Sscanf’serror result, in thiscas e we don’tneed to

becaus e if there was a pro blem, noswitch cas e wi l l match.

The CelsiusFlag func tionbelow wraps itall up. Tothe cal ler,itretur nsapoint ertothe Cel-

sius ﬁeldemb edde d within the celsiusFlag var iable f.The Celsius ﬁeldisthe var iable

that will beupdated bythe Set method dur ing ﬂags pro cessing. The cal l to Var addsthe ﬂag

to the app lic ation’sset ofcommand-line ﬂags, the globalvar iable flag.CommandLine.

Prog ramswit h unusuallycomplex command-lineint erfaces may haveseveral variables ofthis

type.The cal l to Var assig nsa*celsiusFlag argumenttoaflag.Value parameter,causing

thecompi ler toche ckthat *celsiusFlag hasthe necessary met hods.

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

SECTION 7.5. INTERFACE VALUES 181

// CelsiusFlag defines a Celsius flag with the specified name,

// default value, and usage, and returns the address of the flag variable.

// The flag argument must have a quantity and a unit, e.g., "100C".

func CelsiusFlag(name string, value Celsius, usage string) *Celsius {

f:=celsiusFlag{value}

flag.CommandLine.Var(&f, name, usage)

return &f.Celsius

}

No w we can start usingthe new ﬂag in our programs:

gopl.io/ch7/tempflag

var temp = tempconv.CelsiusFlag("temp", 20.0, "the temperature")

func main() {

flag.Parse()

fmt.Println(*temp)

}

Here’satypic al session:

$gobuild gopl.io/ch7/tempflag

$./tempflag

20°C

$./tempflag -temp -18C

-18°C

$./tempflag -temp 212°F

100°C

$./tempflag -temp 273.15K

invalid value "273.15K" for flag -temp: invalid temperature "273.15K"

Usage of ./tempflag:

-temp value

the temperature (default 20°C)

$./tempflag -help

Usage of ./tempflag:

-temp value

the temperature (default 20°C)

Exercis e 7.6: Addsup por t forKelvintemperatures to tempflag.

Exercis e 7.7: Expl ain why the helpmessage contains °C when the defau ltvalue of 20.0 do es

not.

7.5. Int erfaceValues

Conceptu ally, a value ofanint erface typ e,or interface value,has two components, a concrete

type and a value ofthattyp e.These are cal le d theint erface’s dy namic type and dy namic valu e.

Fo r astaticallytyp edlangu agelikeGo, typ es areacompi le-t imecon cept, soatyp e is not a

value.Inour conceptu almodel,aset ofvalues cal le d ty pedes cript o rs prov ide infor mat ion

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

182 CHAPTER 7. INTERFACES

ab out eachtyp e,suchasits nameand methods. Inanint erface value,the typ e comp onent is

repres ente d by the appro priatetyp e des crip tor.

In the fourstatementsbelow,the var iable w takesonthree dif ferentvalues. (Theinitial and

ﬁnalvalues arethe same.)

var w io.Writer

w=os.Stdout

w=new(bytes.Buffer)

w=nil

Let’stakeaclos er lo okatthe value anddynamic beh avior of w af ter eachstatement.The ﬁrs t

st atement declares w:

var w io.Writer

In Go, var iables arealways initialize d to a wel l-deﬁne d value,and int erfaces arenoexception.

Thezerovalue for anint erface has bot h itstyp e andvalue comp onentsset to nil (Figure7.1).

Figure 7.1. Anil int erface value.

An int erface value isdes crib edasnil ornon-ni l basedonits dynamic typ e,sothisisanil

interface value.You can testwhether an interface value isnil using w==nil or w!=nil.

Callingany met hod ofanil int erface value causesapanic:

w.Write([]byte("hello")) // panic: nil pointer dereference

Thesecon d st atement assig nsavalue oftyp e *os.File to w:

w=os.Stdout

Thisassig nmentinv olves an implicitconversionfro m acon crete typ e to anint erface typ e,and

is equivalenttothe explicitconversion io.Writer(os.Stdout).Aconv ersionofthiskind,

whet her explicitorimp licit, capturesthe typ e andthe value ofits operand. The int erface

value’sdynamic typ e is set tothe typ e des crip tor for the point ertyp e *os.File,and its

dy namic value holds a copyof os.Stdout,whichisapointertothe os.File var iable rep-

resent ing the stand ard out put of the pro cess(Figure7.2).

Figure 7.2. An int erface value cont ainingan *os.File pointer.

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

SECTION 7.5. INTERFACE VALUES 183

Callingthe Write method onanint erface value cont ainingan *os.File pointercausesthe

(*os.File).Write method tobecal le d.The cal l pr ints "hello".

w.Write([]byte("hello")) // "hello"

In general,wecannot know at compi letimewhatthe dynamic typ e of anint erface value will

be,soa cal l thro ugh anint erface mustuse dy namic dispatch.Ins teadofadirec t call,the com-

pi ler mustgeneratecodetoobt ain the addressofthe method named Write from the typ e

des crip tor,thenmakeanindirec t call tothataddress. Thereceiverargumentfor the cal l is a

copy ofthe int erface’sdynamic value, os.Stdout.The effec t is asifwehad made thiscal l

direc tly :

os.Stdout.Write([]byte("hello")) // "hello"

Thethirdstatement assig nsavalue oftyp e *bytes.Buffer to the int erface value:

w=new(bytes.Buffer)

Thedynamic typ e is now *bytes.Buffer andthe dynamic value isapoint ertothe newly

al locate d buffer (Figure7.3).

Figure 7.3. An int erface value cont aininga*bytes.Buffer pointer.

Acal l to the Write method usesthe samemechanism as before:

w.Write([]byte("hello")) // writes "hello" to the bytes.Buffer

Thistime, the typ e des crip tor is *bytes.Buffer,sothe (*bytes.Buffer).Write method is

called,wit h theaddressofthe buf fer as the value ofthe receiverparameter.The cal l append s

"hello" to the buf fer.

Final ly, the fourthstatement assig ns nil to the int erface value:

w=nil

Thisres ets both its comp onentsto nil,restoring w to the samestate aswhenitwas decl are d,

whichwas shown inFigure7.1.

An int erface value can holdarbit rar ily large dynamic values. For example, the time.Time

type,whichrepresentsanins tantintime, isastr uct typ e with several unexp orted ﬁelds.Ifwe

create anint erface value fro m it,

var x interface{} = time.Now()

theresultmig htlooklikeFigure7.4. Con ceptu ally, the dynamic value always ﬁts inside the

interface value,nomatterhow large its typ e.(Thisisonlyacon ceptu almodel;arealist ic

implementation isquite dif ferent.)

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

184 CHAPTER 7. INTERFACES

Figure 7.4. An int erface value holdingatime.Time st ruc t.

Interface values may becompare d using == and !=.Two int erface values areequ alifbot h are

ni l,oriftheir dynamic typ es areidenticaland their dynamic values areequ alaccordingtothe

usualbeh avior of == forthattyp e.Because int erface values arecomparable,the y maybeused

as the keysofamap or as the operandofaswitch statement.

Ho wever,iftwo int erface values arecompare d andhavethe samedynamic typ e,but thattyp e

is not comparable (a slice,for ins tance), thenthe comparisonfai lswit h apanic:

var x interface{} = []int{1, 2, 3}

fmt.Println(x == x) // panic: comparing uncomparable type []int

In thisrespect,int erface typ es areunu sual. Other typ es areeit her safelycomparable (like

basic typ es andpoint ers)ornot comparable at all(li keslices, maps, andfunctions), but when

comp aring int erface values oraggregatetyp es that cont ain interface values, wemustbeaware

of the pot ent ial for a panic. A simi lar riskexistswhenusingint erfaces as map keysorswitch

op erands.Onlycompare int erface values if you are cer tain thatthe y cont ain dynamic values

of comparable typ es.

Wh enhandlingerror s,ordur ingdebug ging, itisoften helpf ultorep ort the dynamic typ e of

an interface value.For that, weuse the fmt package’s %T verb:

var w io.Writer

fmt.Printf("%T\n", w) // "<nil>"

w=os.Stdout

fmt.Printf("%T\n", w) // "*os.File"

w=new(bytes.Buffer)

fmt.Printf("%T\n", w) // "*bytes.Buffer"

Internal ly, fmt us esreﬂec tion toobt ain the nameofthe int erface’sdynamic typ e.We’ll lookat

reﬂec tion inChapt er12.

7.5.1. Caveat: An Int erfaceContaining a Nil PointerIsNon-Nil

Anil int erface value,whichcontainsnovalue at all, isnot the sameasanint erface value con-

tainingapoint erthathappens tobenil.Thissubtledistinc tioncre atesatrap into whichevery

Go programmer has stumb led.

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

SECTION 7.5. INTERFACE VALUES 185

Consider the program below.Wit h debug setto true,the main functioncol lec tsthe out put of

thefunction f in a bytes.Buffer.

const debug = true

func main() {

var buf *bytes.Buffer

if debug {

buf = new(bytes.Buffer) // enable collection of output

}

f(buf) // NOTE: subtly incorrect!

if debug {

// ...use buf...

}

// If out is non-nil, output will be written to it.

func f(out io.Writer) {

// ...do something...

if out != nil {

out.Write([]byte("done!\n"))

}

We mig htexp ect thatchang ing debug to false woulddis ablethe col lec tion of the out put,but

in factitcausesthe program topanic dur ingthe out.Write call:

if out != nil {

out.Write([]byte("done!\n")) // panic: nil pointer dereference

}

Wh en main calls f,itassig nsanil point eroftyp e *bytes.Buffer to the out parameter,sothe

dy namic value of out is nil.How ever, its dynamic typ e is *bytes.Buffer,meaningthat out

is a non-ni l interface cont aininganil point ervalue (Figure7.5), sothe defensive che ck

out != nil is still true.

Figure 7.5. Anon-ni l interface cont aininganil point er.

As before, the dynamic disp atch mechanism deter mines that (*bytes.Buffer).Write mu st

be cal le d butthistimewit h areceivervalue thatisnil.For som e types, suchas *os.File, nil

is a valid receiver(§6.2.1), but *bytes.Buffer is not among them. Themet hod iscal le d,but

it panics as ittries toaccessthe buf fer.

Thepro blem isthatalt hough a nil *bytes.Buffer pointerhas the methodsneeded tosat isf y

theint erface,itdoesn’tsat isf y the behavioral re quirements ofthe int erface.Inpar tic ular, the

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

186 CHAPTER 7. INTERFACES

call violatesthe imp licitpre con dit ion of (*bytes.Buffer).Write that its receiverisnot nil,

so assig ningthe nil point ertothe int erface was a mist ake . Thesolut ion istochange the typ e

of buf in main to io.Writer,there byavoidingthe assig nmentofthe dysf unctionalvalue to

theint erface in the ﬁrs t pl ace:

var buf io.Writer

if debug {

buf = new(bytes.Buffer) // enable collection of output

}

f(buf) // OK

No w that we’ve cov ere d themechanics ofint erface values, let’stakealook at som e more

importantint erfaces fro m Go’sstand ard librar y.Inthe next three sec tion s,we’ll see how int er-

faces areusedfor sor ting, web ser ving, and error handling.

7.6. Sorting with sort.Interface

Like str ing for matting , sortingisafre quentlyusedoperat ioninmanyprograms. Alt hough a

minimal Quicks ort can bewritt eninabout 15lines, a robustimp lementation ismuchlon g er,

anditisnot the kindofcodeweshouldwishtowrite ane w or copyeachtimeweneed it.

Fo rtunate ly, the sort packageprovides in-place sor tingofany sequence accordingtoany

order ingfunction. Its designisrat her unusual. Inmanylangu ages, the sor tingalgor it hmis

asso ciate d with thesequence dat a type,whi le theorder ingfunctionisass oci ated wit h thetyp e

of the elements. Bycontrast, Go’s sort.Sort func tionassumesnot hingabout the represen-

tation of eit her the sequence orits elements. Ins tead, itusesanint erface, sort.Interface,to

sp ecif y thecontrac t betweenthe gener ic sort algor it hmand eachsequence typ e that may be

sorted.Animp lementation of thisint erface deter mines bot h thecon crete represent ation of

thesequence,whichisoften a slice,and the desired order ingofits elements.

An in-place sor t algor it hmneedsthree things—thelengt h of the sequence,ameans ofcom-

paring two elements, andaway toswap two elements—so the y arethe three met hodsof

sort.Interface:

package sort

type Interface interface {

Len() int

Less(i, j int) bool // i, j are indices of sequence elements

Swap(i, j int)

}

To sor t anysequence,weneed todeﬁne a typ e that implements these three met hods, then

apply sort.Sort to anins tance ofthattyp e.Asperhaps the simplestexample, con sider

sortingaslice ofstr ings. Thenew typ e StringSlice andits Len, Less,and Swap methodsare

shown below.

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

SECTION 7.6. SORTING WITH SORT.INTERFACE 187

type StringSlice []string

func (p StringSlice) Len() int {return len(p) }

func (p StringSlice) Less(i, j int) bool { return p[i] < p[j] }

func (p StringSlice) Swap(i, j int) {p[i], p[j] = p[j], p[i] }

No w we can sor t aslice ofstr ings, names,byconvertingthe slice toaStringSlice li kethis:

sort.Sort(StringSlice(names))

Theconversionyieldsaslice value wit h thesamelengt h, capacity,and underly ing array as

names butwit h atyp e that has the three met hodsrequired for sor ting.

Sortingaslice ofstr ingsissocommonthatthe sort packageprovides the StringSlice type,

as wel l as a functioncal le d Strings so thatthe cal l ab ove can besimpliﬁed to

sort.Strings(names).

Thetechnique hereiseasi lyadapt edtoother sor t orders, for ins tance,toignorecapit alizat ion

or speci al ch arac ters. (TheGoprogram thatsor tsindex ter msand pagenumbers for this

bookdoesthis, wit h ext ra log ic forRom annumerals.) For morecomplic ated sor ting, weuse

thesameide a, butwit h more complic ated dat a st ruc tures ormorecomplic ated imp lemen-

tation s of the sort.Interface methods.

Ourrunningexamplefor sor tingwill beamusic playlist, displaye d as a table.Eachtrackisa

singlerow,and eachcolumn isanatt ribut e of thattrack, like artist, tit le, and runningtime.

Im agine thatagraphic al us erint erface presentsthe table,and thatclicking the head ofacol-

umn causesthe playlisttobesor ted bythatatt ribut e;clicking the samecolumn headagain

re versesthe order.Let’slook at whatmig hthappen in respons e to eachclick.

Thevar iable tracks belowcontainsaplaylist. (Oneofthe aut hor s ap olog izes forthe other

author’smusic al tastes.) Eachelementisindirec t,apoint ertoaTrack.Alt hough the code

belowwou ldwor k if westore d the Tracksdirec tly,the sor t func tionwill swap manypairsof

elements, soitwill run fasterifeachelementisapoint er, whichisasinglemachineword,

instead of an ent ire Track,whichmig htbeeig htwords ormore.

gopl.io/ch7/sorting

type Track struct {

Title string

Artist string

Album string

Year int

Length time.Duration

}

var tracks = []*Track{

{"Go", "Delilah", "From the Roots Up", 2012, length("3m38s")},

{"Go", "Moby", "Moby", 1992, length("3m37s")},

{"Go Ahead", "Alicia Keys", "As I Am", 2007, length("4m36s")},

{"Ready 2 Go", "Martin Solveig", "Smash", 2011, length("4m24s")},

}

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

188 CHAPTER 7. INTERFACES

func length(s string) time.Duration {

d, err := time.ParseDuration(s)

if err != nil {

panic(s)

}

return d

}

The printTracks func tionprintsthe playlist as a table.Agraphic al displaywou ldbenicer,

butthislit tle routine usesthe text/tabwriter packagetopro duce a table whose columnsare

ne atlyalig ned and padde d as shown below.Obs erve that *tabwriter.Writer satisﬁes

io.Writer.Itcol lec tseachpie ce of dat a wr itt entoit; its Flush method for mats the ent ire ta-

bleand writesitto os.Stdout.

func printTracks(tracks []*Track) {

const format = "%v\t%v\t%v\t%v\t%v\t\n"

tw := new(tabwriter.Writer).Init(os.Stdout, 0, 8, 2, '',0)

fmt.Fprintf(tw, format, "Title", "Artist", "Album", "Year", "Length")

fmt.Fprintf(tw, format, "-----", "------", "-----", "----", "------")

for _, t := range tracks {

fmt.Fprintf(tw, format, t.Title, t.Artist, t.Album, t.Year, t.Length)

}

tw.Flush() // calculate column widths and print table

}

To sor t theplaylistbythe Artist ﬁeld, wedeﬁne a newslice typ e with thenecessary Len,

Less,and Swap methods, analogou s to whatwedid for StringSlice.

type byArtist []*Track

func (x byArtist) Len() int {return len(x) }

func (x byArtist) Less(i, j int) bool { return x[i].Artist < x[j].Artist }

func (x byArtist) Swap(i, j int) {x[i], x[j] = x[j], x[i] }

To cal l thegener ic sort routine,wemustﬁrs t conv ert tracks to the newtyp e, byArtist,that

deﬁnesthe order :

sort.Sort(byArtist(tracks))

Af ter sor tingthe slice byartist, the out put fro m printTracks is

Title Artist Album Year Length

----- ------ ----- ---- ------

Go Ahead Alicia Keys As I Am 2007 4m36s

Go Delilah From the Roots Up 2012 3m38s

Ready 2 Go Martin Solveig Smash 2011 4m24s

Go Moby Moby 1992 3m37s

If the userrequests ‘‘sort byartist’’ asecon d time,we’ll sor t thetracks in reverse.Weneedn’t

deﬁne a newtyp e byReverseArtist with aninv erted Less method,how ever, since the sort

packageprovides a Reverse func tionthattransfor msany sor t order toits invers e.

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

SECTION 7.6. SORTING WITH SORT.INTERFACE 189

sort.Sort(sort.Reverse(byArtist(tracks)))

Af ter reverse-s ortingthe slice byartist, the out put fro m printTracks is

Title Artist Album Year Length

----- ------ ----- ---- ------

Go Moby Moby 1992 3m37s

Ready 2 Go Martin Solveig Smash 2011 4m24s

Go Delilah From the Roots Up 2012 3m38s

Go Ahead Alicia Keys As I Am 2007 4m36s

The sort.Reverse func tiondes ervesaclos er lo oksince itusescomposition (§6.3), whichis

an importantide a. The sort packagedeﬁnesanunexp orted typ e reverse,whichisastr uct

that embeds a sort.Interface.The Less method for reverse callsthe Less method ofthe

embedde d sort.Interface value,but wit h theindices ﬂip ped,reversingthe order ofthe sor t

resu lts.

package sort

type reverse struct{ Interface } // that is, sort.Interface

func (r reverse) Less(i, j int) bool { return r.Interface.Less(j, i) }

func Reverse(data Interface) Interface { return reverse{data} }

Len and Swap,the other two met hodsof reverse,are imp licitlyprovide d by the original

sort.Interface value because itisanemb edde d ﬁeld. The exp orted function Reverse

returns an ins tance ofthe reverse type thatcontainsthe original sort.Interface value.

To s or t by a dif ferentcolumn, wemustdeﬁne a newtyp e,suchas byYear:

type byYear []*Track

func (x byYear) Len() int {return len(x) }

func (x byYear) Less(i, j int) bool { return x[i].Year < x[j].Year }

func (x byYear) Swap(i, j int) {x[i], x[j] = x[j], x[i] }

Af ter sor ting tracks by yearusing sort.Sort(byYear(tracks)), printTracks shows a

chro nolog ical list ing:

Title Artist Album Year Length

----- ------ ----- ---- ------

Go Moby Moby 1992 3m37s

Go Ahead Alicia Keys As I Am 2007 4m36s

Ready 2 Go Martin Solveig Smash 2011 4m24s

Go Delilah From the Roots Up 2012 3m38s

Fo r ever y slice elementtyp e andevery order ingfunctionweneed,wedeclare a new imp le-

ment ation of sort.Interface.Asyou can see,the Len and Swap methodshaveidenticaldef-

inition s forall slice typ es. In the next example, the con crete typ e customSort combines a slice

with a func tion, letting usdeﬁne a newsor t order bywriting onlythe comparisonfunction.

In cidentally, the con crete typ es that implement sort.Interface arenot always slices; cus-

tomSort is a str uct typ e.

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

190 CHAPTER 7. INTERFACES

type customSort struct {

t[]*Track

less func(x, y *Track) bool

}

func (x customSort) Len() int {return len(x.t) }

func (x customSort) Less(i, j int) bool { return x.less(x.t[i], x.t[j]) }

func (x customSort) Swap(i, j int) {x.t[i], x.t[j] = x.t[j], x.t[i] }

Let’sdeﬁne a multi-t ier order ingfunctionwhose primary sor t ke y is the Title,whose

second ary key isthe Year,and whose ter tiary key isthe runningtime, Length.Here’sthe cal l

to Sort using an anony mou s order ingfunction:

sort.Sort(customSort{tracks, func(x, y *Track) bool {

if x.Title != y.Title {

return x.Title < y.Title

}

if x.Year != y.Year {

return x.Year < y.Year

}

if x.Length != y.Length {

return x.Length < y.Length

}

return false

}})

Andhere’sthe result. Not ice thatthe tie bet weenthe two tracks tit led ‘‘Go’’ is bro ken in favor

of the older one.

Title Artist Album Year Length

----- ------ ----- ---- ------

Go Moby Moby 1992 3m37s

Go Delilah From the Roots Up 2012 3m38s

Go Ahead Alicia Keys As I Am 2007 4m36s

Ready 2 Go Martin Solveig Smash 2011 4m24s

Although sor tingasequence oflengt h n re quires O(n log n)comparisonoperat ions,testing

whet her a sequence isalready sor ted requires at most n−1comparisons.The IsSorted func-

tion fro m the sort packageche cks thisfor us. Like sort.Sort,itabstrac ts both thesequence

andits order ingfunctionusing sort.Interface,but it never cal lsthe Swap method:This

co de demon strates the IntsAreSorted and Ints func tions and the IntSlice type:

values := []int{3, 1, 4, 1}

fmt.Println(sort.IntsAreSorted(values)) // "false"

sort.Ints(values)

fmt.Println(values) // "[1 1 3 4]"

fmt.Println(sort.IntsAreSorted(values)) // "true"

sort.Sort(sort.Reverse(sort.IntSlice(values)))

fmt.Println(values) // "[4 3 1 1]"

fmt.Println(sort.IntsAreSorted(values)) // "false"

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

SECTION 7.7. THE HTTP.HANDLER INTERFACE 191

Fo r conv enience,the sort packageprovides versions ofits functions and typ es sp eci alizedfor

[]int, []string,and []float64 usingtheir natural order ings. For other typ es, such as

[]int64 or []uint,we’re onour own,thoug h thepat h is short.

Exercis e 7.8: Many GUIsprovide a table widget wit h astatefulmulti-t ier sort:the primary

sort key isthe most recentlyclicke d column head, the secon dar y sort key isthe secon d-most

re centlyclicke d column head, and soon. Deﬁne animp lementation of sort.Interface for

us e by suchatable.Compare thatappro ach wit h repeated sor tingusing sort.Stable.

Exercis e 7.9: Us e the html/template package(§4.6) toreplace printTracks with a func tion

that displays the tracks as an HTML table.Use the solut ion tothe pre vious exercis e to arrange

that eachclickonacolumn headmakes an HTTP requesttosor t thetable.

Exercis e 7.10: The sort.Interface type can beadapt edtoother uses. Write a function

IsPalindrome(s sort.Interface) bool that rep ortswhether the sequence s is a palin-

drom e,inother words,reversingthe sequence wou ldnot change it. Assume thatthe elements

at indices i and j areequ alif !s.Less(i, j) && !s.Less(j, i).

7.7. The http.Handler Int erface

In Chapt er1,wesaw a glimps e of how touse the net/http packagetoimp lementweb clients

(§1.5) andser vers(§1.7). Inthissec tion,we’ll lookmoreclos ely atthe ser ver API, whose

foundation isthe http.Handler interface:

net/http

package http

type Handler interface {

ServeHTTP(w ResponseWriter, r *Request)

}

func ListenAndServe(address string, h Handler) error

The ListenAndServe func tionrequires a ser ver address, suchas "localhost:8000",and an

inst anceofthe Handler interface towhichall requests shouldbedispatch ed. Itrunsforever,

or unt i l theser ver fails (or fai lstostart)wit h an erro r,always non-ni l,whichitretur ns.

Im agine ane-commerce site wit h adat abas e mappingthe items for sale totheir prices in dol-

lars.The program below shows the simplestimaginableimp lementation.Itmodel s theinv en-

tory asamap typ e, database,towhichwe’ve att ach edaServeHTTP method sothatitsat isﬁes

the http.Handler interface.The handler rangesoverthe map andprintsthe items.

gopl.io/ch7/http1

func main() {

db := database{"shoes": 50, "socks": 5}

log.Fatal(http.ListenAndServe("localhost:8000", db))

}

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

192 CHAPTER 7. INTERFACES

type dollars float32

func (d dollars) String() string { return fmt.Sprintf("$%.2f", d) }

type database map[string]dollars

func (db database) ServeHTTP(w http.ResponseWriter, req *http.Request) {

for item, price := range db {

fmt.Fprintf(w, "%s: %s\n", item, price)

}

If westart the ser ver,

$gobuild gopl.io/ch7/http1

$./http1 &

then connec t to itwit h the fetch prog ram from Sec tion 1.5 (oraweb brows er if you prefer),

we getthe fol low ing out put:

$gobuild gopl.io/ch1/fetch

$./fetch http://localhost:8000

shoes: $50.00

socks: $5.00

So far,the ser ver can onlylistits entire inv ent ory and will dothisfor every request, regardless

of URL. A more realist icser ver deﬁnesmultipledif ferentURLs, eachtrigger ingadif ferent

behavior.Let’scal l theexist ing one /list andadd anot her one cal le d /price that rep ortsthe

pr ice ofasingleitem, speciﬁed asarequestparameter like /price?item=socks.

gopl.io/ch7/http2

func (db database) ServeHTTP(w http.ResponseWriter, req *http.Request) {

switch req.URL.Path {

case "/list":

for item, price := range db {

fmt.Fprintf(w, "%s: %s\n", item, price)

}

case "/price":

item := req.URL.Query().Get("item")

price, ok := db[item]

if !ok {

w.WriteHeader(http.StatusNotFound) // 404

fmt.Fprintf(w, "no such item: %q\n", item)

return

}

fmt.Fprintf(w, "%s\n", price)

default:

w.WriteHeader(http.StatusNotFound) // 404

fmt.Fprintf(w, "no such page: %s\n", req.URL)

}

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

SECTION 7.7. THE HTTP.HANDLER INTERFACE 193

No w thehandler decides whatlog ic to exe cut e basedonthe pat h comp onent ofthe URL,

req.URL.Path.Ifthe handler doesn’trecog nize thepat h, it rep ortsanHTTPerror tothe

clientbycal ling w.WriteHeader(http.StatusNotFound);thismustbedon e before writing

anytext to w.(In cidentally, http.ResponseWriter is another interface.Itaug ments

io.Writer with met hodsfor sendingHTTPrespons e he aders.) Equivalently, wecou lduse

the http.Error ut i lit y func tion:

msg := fmt.Sprintf("no such page: %s\n", req.URL)

http.Error(w, msg, http.StatusNotFound) // 404

Thecas e for /price callsthe URL’s Query method toparse the HTTPrequestparametersasa

map, ormorepre cis ely,amultimapoftyp e url.Values (§6.2.1) fro m the net/url package. It

then ﬁnd s theﬁrs t item parameter andlooks upits price.Ifthe itemwasn’tfound,itrep orts

an erro r.

Here’s an examplesessionwit h thenew server :

$gobuild gopl.io/ch7/http2

$gobuild gopl.io/ch1/fetch

$./http2 &

$./fetch http://localhost:8000/list

shoes: $50.00

socks: $5.00

$./fetch http://localhost:8000/price?item=socks

$5.00

$./fetch http://localhost:8000/price?item=shoes

$50.00

$./fetch http://localhost:8000/price?item=hat

no such item: "hat"

$./fetch http://localhost:8000/help

no such page: /help

Obviou sly wecou ldkeepaddingcas es to ServeHTTP,but inarealist icapp lic ation, it’scon-

venienttodeﬁne the log ic foreachcas e in a sep aratefunctionormet hod.Fur thermore,

re lated URLs may need simi lar logic; several image ﬁles may haveURLs ofthe for m

/images/*.png,for ins tance.For these reasons, net/http prov ides ServeMux,are quest

mu ltipl exer,tosimplif y theass oci ationbet weenURLs andhandlers. A ServeMux ag gregates a

collec tion of http.Handlersint o asingle http.Handler.Again, wesee thatdif ferenttyp es

satisf yingthe sameint erface are substitutabl e:the web ser ver can disp atch requests toany

http.Handler,regardlessofwhichcon crete typ e is behindit.

Fo r amorecomplex applic ation, several ServeMuxes may becomposedtohandlemore

intr icate dispatchingrequirements. Godoesn’thaveacanonic al we b framewor k analogou s to

Ru by’sRai lsorPyt hon’sDjango.Thisisnot tosay thatsuchframewor ksdon’texist,but the

buildingblo cks in Go’sstand ard librar y are ﬂexibleenoug h that framewor ksare often

unnecessary.Fur thermore, alt hough framewor ksare convenientinthe early phas es of a

proj e ct, their addition alcomplexity canmakelon g er-term maintenanceharder.

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

194 CHAPTER 7. INTERFACES

In the program below,wecre ate a ServeMux anduse ittoass oci atethe URLs wit h thecor-

resp ondinghandlersfor the /list and /price op erat ions,whichhavebeensplit int o separate

methods. Wethenuse the ServeMux as the main handler in the cal l to ListenAndServe.

gopl.io/ch7/http3

func main() {

db := database{"shoes": 50, "socks": 5}

mux := http.NewServeMux()

mux.Handle("/list", http.HandlerFunc(db.list))

mux.Handle("/price", http.HandlerFunc(db.price))

log.Fatal(http.ListenAndServe("localhost:8000", mux))

}

type database map[string]dollars

func (db database) list(w http.ResponseWriter, req *http.Request) {

for item, price := range db {

fmt.Fprintf(w, "%s: %s\n", item, price)

}

func (db database) price(w http.ResponseWriter, req *http.Request) {

item := req.URL.Query().Get("item")

price, ok := db[item]

if !ok {

w.WriteHeader(http.StatusNotFound) // 404

fmt.Fprintf(w, "no such item: %q\n", item)

return

}

fmt.Fprintf(w, "%s\n", price)

}

Let’sfocus onthe two cal lsto mux.Handle that reg ister the handlers. Inthe ﬁrs t on e, db.list

is a met hod value (§6.4), thatis, a value oftyp e

func(w http.ResponseWriter, req *http.Request)

that, whencal le d,inv okesthe database.list method wit h thereceivervalue db.So db.list

is a functionthatimp lements handler-li kebeh avior,but since ithas nomet hods, itdoesn’tsat-

isfy the http.Handler interface andcan’tbepasseddirec tly to mux.Handle.

Theexpression http.HandlerFunc(db.list) is a conversion, not a functioncal l,since

http.HandlerFunc is a typ e.Ithas the fol low ing deﬁnition:

net/http

package http

type HandlerFunc func(w ResponseWriter, r *Request)

func (f HandlerFunc) ServeHTTP(w ResponseWriter, r *Request) {

f(w, r)

}

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

SECTION 7.7. THE HTTP.HANDLER INTERFACE 195

HandlerFunc demon strates som e unusualfeaturesofGo’sint erface mechanism. Itisafunc-

tion typ e that has met hodsand sat isﬁes an interface, http.Handler.The beh avior of its

ServeHTTP method istocal l theunderly ing function. HandlerFunc is thu s an adapterthat

lets a functionvalue sat isf y an interface,where the functionand the int erface’ssolemet hod

have the samesig nature. Ineffec t,thistrick lets a singletyp e such as database satisf y the

http.Handler interface several different ways: oncethrough its list method,oncethrough

its price method,and soon.

Becaus e register ingahandler thisway issocommon, ServeMux hasaconvenience met hod

called HandleFunc that does itfor us, sowecan simplif y thehandler reg ist rat ioncodetothis:

gopl.io/ch7/http3a

mux.HandleFunc("/list", db.list)

mux.HandleFunc("/price", db.price)

It’seasy tosee fro m thecodeabove how one wou ldcon str uct a program in whichthere are

twodif ferentweb ser vers, listeningondif ferentpor ts, deﬁningdif ferentURLs, anddis-

patchingtodif ferenthandlers. Wewou ldjustcon str uct another ServeMux andmakeanother

call to ListenAndServe,perhaps conc urrently. But inmostprograms, one web ser ver is

plenty.Als o,it’stypic al to deﬁne HTTPhandlersacrossmany ﬁles ofanapp lic ation, andit

wouldbeanuisance if the y al l hadtobeexplicitlyreg istered wit h theapp lic ation’s ServeMux

inst ance.

So,for convenience, net/http prov ides a global ServeMux inst ancecal le d DefaultServeMux

andpackage-le vel functions cal le d http.Handle and http.HandleFunc.Touse Default-

ServeMux as the ser ver’smain handler,weneedn’tpassitto ListenAndServe; nil wi l l do.

Theser ver’smain functioncan thenbesimpliﬁed to

gopl.io/ch7/http4

func main() {

db := database{"shoes": 50, "socks": 5}

http.HandleFunc("/list", db.list)

http.HandleFunc("/price", db.price)

log.Fatal(http.ListenAndServe("localhost:8000", nil))

}

Final ly, animp ortantreminder : as wemention edinSec tion 1.7, the web ser ver invo kes each

hand ler in a new goroutine,sohandlersmusttakepre caution s such as locki n g when accessing

var iables thatother goroutines, includingother requests tothe samehandler,may beaccess-

ing. We’ll tal k ab out con cur rency inthe next two chapt ers.

Exercis e 7.11: Addaddition alhandlerssothatclients can create, read, update, and delete

database ent ries. For example, a requestofthe for m /update?item=socks&price=6 wi l l

up datethe price ofaniteminthe inv ent ory and rep ort anerror ifthe itemdoesnot exist orif

theprice isinvalid.(Warning: thischange int roduces conc urrentvar iable updates.)

Exercis e 7.12: Change the handler for /list to print its out put asanHTMLtable,not text.

Yo u mayﬁnd the html/template package(§4.6) useful.

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

196 CHAPTER 7. INTERFACES

7.8. The error Int erface

Sincethe beg inningofthisbook, we’ve beenusingand cre ating values ofthe mysteriou s

predeclare d error type wit houtexplainingwhatitreallyis. Infac t, it’sjustanint erface typ e

with a singlemet hod thatretur ns an error message:

type error interface {

Error() string

}

Thesimplestway tocre ate an error is bycal ling errors.New,whichretur nsanew error for

agiven erro r mess age . Theent ire errors packageisonlyfourlines long:

package errors

func New(text string) error { return &errorString{text} }

type errorString struct { text string }

func (e *errorString) Error() string { return e.text }

Theunderly ing typ e of errorString is a str uct,not a str ing , to pro tec t itsrepresent ation fro m

inadvertent(or premedit ate d)updates. And the reasonthatthe point ertyp e *errorString,

not errorString alone,sat isﬁes the error interface issothatevery cal l to New al locatesadis-

tinc t error inst ancethatisequ altonoother.Wewou ldnot wantadistinguishe d er ror such

as io.EOF to compare equ altoone thatmerelyhappene d to havethe samemessage .

fmt.Println(errors.New("EOF") == errors.New("EOF")) // "false"

Callsto errors.New arerel ative lyinf requentbecause there’saconvenientwrapperfunction,

fmt.Errorf,thatdoesstr ing for matting too.Weuseditseveral times in Chapt er5.

package fmt

import "errors"

func Errorf(format string, args ...interface{}) error {

return errors.New(Sprintf(format, args...))

}

Although *errorString maybethe simplesttyp e of error,itisfar fro m theonlyone.For

example, the syscall packageprovides Go’slow-le vel systemcal l API. On many platfor ms, it

deﬁnesanumerictyp e Errno that sat isﬁes error,and onUnix platfor ms, Errno’s Error

method doesalooku p in a table ofstr ings, as shown below :

package syscall

type Errno uintptr // operating system error code

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

SECTION 7.9. EXAMPLE: EXPRESSION EVALUATOR197

var errors = [...]string{

1: "operation not permitted", // EPERM

2: "no such file or directory", // ENOENT

3: "no such process", // ESRCH

// ...

}

func (e Errno) Error() string {

if 0 <= int(e) && int(e) < len(errors) {

return errors[e]

}

return fmt.Sprintf("errno %d", e)

}

Thefol low ing statement cre atesanint erface value holdingthe Errno value 2,sig nifying the

POSIX ENOENT condit ion:

var err error = syscall.Errno(2)

fmt.Println(err.Error()) // "no such file or directory"

fmt.Println(err) // "no such file or directory"

Thevalue of err is shown graphic ally inFigure7.6.

Figure 7.6. An int erface value holdingasyscall.Errno integer.

Errno is anefﬁcientrepresent ation of systemcal l er ror s draw n from a ﬁnite set, anditsat isﬁes

thestand ard error interface.We’ll see other typ es that sat isf y this int erface in Sec tion 7.11.

7.9. Example: Expression Evaluator

In thissec tion,we’ll bui ld an evaluatorfor simplearithmeticexpressions.We’ll use anint er-

face, Expr,torepresent any expressioninthislangu age. For now,thisint erface needsno

methods, but we’ll add som e later.

// An Expr is an arithmetic expression.

type Expr interface{}

Ourexpressionlangu agecon sists of ﬂoating-p ointlit erals; the binar y op erator s +, -, *,and /;

theunary operator s -x and +x;functioncal ls pow(x,y), sin(x),and sqrt(x);var iables such

as x and pi;and ofcours e parent hes es andstand ard operator precedence.All values areof

type float64.Hereare som e exampleexpressions:

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

198 CHAPTER 7. INTERFACES

sqrt(A / pi)

pow(x, 3) + pow(y, 3)

(F - 32) * 5 / 9

The ﬁve con crete typ es belowrepresent par tic ularkinds ofexpression. A Var repres ents a ref-

erence toavar iable.(We’ll soonsee why itisexp orted.) A literal repres ents a ﬂoating-

pointcon stant.The unary and binary typesrepresent operator expressions wit h on e or two

op erands,whichcan beany kindof Expr.Acall repres ents a functioncal l; we’ll restr ict its

fn ﬁeldto pow, sin,or sqrt.

gopl.io/ch7/eval

// A Var identifies a variable, e.g., x.

type Var string

// A literal is a numeric constant, e.g., 3.141.

type literal float64

// A unary represents a unary operator expression, e.g., -x.

type unary struct {

op rune // one of '+', '-'

xExpr

}

// A binary represents a binary operator expression, e.g., x+y.

type binary struct {

op rune // one of '+', '-', '*', '/'

x, y Expr

}

// A call represents a function call expression, e.g., sin(x).

type call struct {

fn string // one of "pow", "sin", "sqrt"

args []Expr

}

To evaluate anexpressioncontainingvar iables, we’ll need an enviro nment that maps variable

namestovalues:

type Env map[Var]float64

We’ll als o ne e d each kindofexpressiontodeﬁne an Eval method thatretur nsthe expression’s

value in a given enviro nment.Since every expressionmustprovide thismet hod,weadd itto

the Expr interface.The packageexp ortsonlythe typ es Expr, Env,and Var;clients can use the

evaluatorwit houtaccesstothe other expressiontyp es.

type Expr interface {

// Eval returns the value of this Expr in the environment env.

Eval(env Env) float64

}

Thecon crete Eval methodsare shown below.The method for Var perfor msanenv iro nment

lo oku p,whichretur nszeroifthe var iable isnot deﬁne d,and the method for literal simply

returnsthe lit eral value.

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

SECTION 7.9. EXAMPLE: EXPRESSION EVALUATOR199

func (v Var) Eval(env Env) float64 {

return env[v]

}

func (l literal) Eval(_ Env) float64 {

return float64(l)

}

The Eval methodsfor unary and binary re cursive lyevaluate their operands,thenapp l y the

op erat ion op to them. Wedon’tcon sider divisions byzeroorinﬁnity tobeerror s,since the y

produce a result, albeit non-ﬁnite. Final ly, the method for call evaluatesthe arguments tothe

pow, sin,or sqrt func tion, thencal lsthe cor respondingfunction in the math package.

func (u unary) Eval(env Env) float64 {

switch u.op {

case '+':

return +u.x.Eval(env)

case '-':

return -u.x.Eval(env)

}

panic(fmt.Sprintf("unsupported unary operator: %q", u.op))

}

func (b binary) Eval(env Env) float64 {

switch b.op {

case '+':

return b.x.Eval(env) + b.y.Eval(env)

case '-':

return b.x.Eval(env) - b.y.Eval(env)

case '*':

return b.x.Eval(env) * b.y.Eval(env)

case '/':

return b.x.Eval(env) / b.y.Eval(env)

}

panic(fmt.Sprintf("unsupported binary operator: %q", b.op))

}

func (c call) Eval(env Env) float64 {

switch c.fn {

case "pow":

return math.Pow(c.args[0].Eval(env), c.args[1].Eval(env))

case "sin":

return math.Sin(c.args[0].Eval(env))

case "sqrt":

return math.Sqrt(c.args[0].Eval(env))

}

panic(fmt.Sprintf("unsupported function call: %s", c.fn))

}

Several ofthese met hodscan fail. For example, a call expressioncou ldhaveanunknown

func tionorthe wro ngnumberofarguments. It’sals o possible tocon str uct a unary or binary

expressionwit h an invalid operator suchas ! or < (although the Parse func tionmention ed

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

200 CHAPTER 7. INTERFACES

belowwill never do this). Thes e er ror s caus e Eval to panic. Other erro rs, like evaluating a

Var notpresent inthe env iro nment,merelycause Eval to retur n thewro ngresult. All ofthese

er ror s couldbedetec ted byins pec ting the Expr before evaluating it. That will bethe job ofthe

Check method,whichwewill showsoon, but ﬁrs t let’stest Eval.

The TestEval func tionbelow isa testofthe evaluator. Itusesthe testing package, which

we’ll explain in Chapt er11, but for now it’senoug h to knowthatcal ling t.Errorf reportsan

er ror.The functionloops overatable ofinp uts thatdeﬁnesthree expressions and dif ferent

enviro nmentsfor eachone.The ﬁrs t expressioncomputesthe radius ofacirclegiven its area

A,the secon d comp utesthe sum ofthe cub esoftwo var iables x and y,and the thirdconvertsa

Fahren heittemperature F to Cel siu s.

func TestEval(t *testing.T) {

tests := []struct {

expr string

env Env

want string

}{

{"sqrt(A / pi)", Env{"A": 87616, "pi": math.Pi}, "167"},

{"pow(x, 3) + pow(y, 3)", Env{"x": 12, "y": 1}, "1729"},

{"pow(x, 3) + pow(y, 3)", Env{"x": 9, "y": 10}, "1729"},

{"5 / 9 * (F - 32)", Env{"F": -40}, "-40"},

{"5 / 9 * (F - 32)", Env{"F": 32}, "0"},

{"5 / 9 * (F - 32)", Env{"F": 212}, "100"},

}

var prevExpr string

for _, test := range tests {

// Print expr only when it changes.

if test.expr != prevExpr {

fmt.Printf("\n%s\n", test.expr)

prevExpr = test.expr

}

expr, err := Parse(test.expr)

if err != nil {

t.Error(err) // parse error

continue

}

got := fmt.Sprintf("%.6g", expr.Eval(test.env))

fmt.Printf("\t%v => %s\n", test.env, got)

if got != test.want {

t.Errorf("%s.Eval() in %s = %q, want %q\n",

test.expr, test.env, got, test.want)

}

Fo r each ent ryinthe table,the testparsesthe expression, evaluatesitinthe env iro nment,and

pr intsthe result. Wedon’thavespace toshowthe Parse func tionhere, but you’ll ﬁnd itifyou

down loadthe packageusing go get.

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

SECTION 7.9. EXAMPLE: EXPRESSION EVALUATOR201

The go test command(§11.1) runsapackage’stests:

$gotest -v gopl.io/ch7/eval

The -v ﬂag lets ussee the print edout put of the test, whichisnor mal lysup pressedfor a suc-

cessf ultestlikethisone.Hereisthe out put of the test’s fmt.Printf st atements:

sqrt(A / pi)

map[A:87616 pi:3.141592653589793] => 167

pow(x, 3) + pow(y, 3)

map[x:12 y:1] => 1729

map[x:9 y:10] => 1729

5/9*(F-32)

map[F:-40] => -40

map[F:32] => 0

map[F:212] => 100

Fo rtunate lythe inp uts sofar haveall beenwel l formed,but our luckisunlikelytolast. Evenin

interprete d languages, itiscommontoche ckthe syntaxfor stati c er ror s,thatis, mist akesthat

canbedetec ted wit houtrunningthe program. Bysep arat ingthe staticche cks fro m the

dy namic ones, wecan detec t er ror s so onerand per for m many che cks onlyonceins teadof

each time an expressionisevaluate d.

Let’sadd anot her met hod tothe Expr interface.The Check method che cks for staticerror s in

an expressionsyntaxtre e.We’ll explain its vars parameter in a mom ent.

type Expr interface {

Eval(env Env) float64

// Check reports errors in this Expr and adds its Vars to the set.

Check(vars map[Var]bool) error

}

Thecon crete Check methodsare shown below.Evaluation of literal and Var cannot fai l,so

the Check methodsfor these typ es return nil.The methodsfor unary and binary ﬁrs t ch eck

that the operator isvalid,thenrec ursivelyche ckthe operands.Simi larly,the method for call

ﬁrs t ch ecksthatthe functionisknown and has the rig htnumberofarguments, thenrec ur-

sive lyche cks eachargument.

func (v Var) Check(vars map[Var]bool) error {

vars[v] = true

return nil

}

func (literal) Check(vars map[Var]bool) error {

return nil

}

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

202 CHAPTER 7. INTERFACES

func (u unary) Check(vars map[Var]bool) error {

if !strings.ContainsRune("+-", u.op) {

return fmt.Errorf("unexpected unary op %q", u.op)

}

return u.x.Check(vars)

}

func (b binary) Check(vars map[Var]bool) error {

if !strings.ContainsRune("+-*/", b.op) {

return fmt.Errorf("unexpected binary op %q", b.op)

}

if err := b.x.Check(vars); err != nil {

return err

}

return b.y.Check(vars)

}

func (c call) Check(vars map[Var]bool) error {

arity, ok := numParams[c.fn]

if !ok {

return fmt.Errorf("unknown function %q", c.fn)

}

if len(c.args) != arity {

return fmt.Errorf("call to %s has %d args, want %d",

c.fn, len(c.args), arity)

}

for _, arg := range c.args {

if err := arg.Check(vars); err != nil {

return err

}

return nil

}

var numParams = map[string]int{"pow": 2, "sin": 1, "sqrt": 1}

We’ve liste d aselec tion of ﬂawed inp uts andthe error s they elicit, in two gro ups. The Parse

func tion(notshown)rep ortssyntaxerror s andthe Check func tionrep ortssemanticerror s.

x%2 unexpected '%'

math.Pi unexpected '.'

!true unexpected '!'

"hello" unexpected '"'

log(10) unknown function "log"

sqrt(1, 2) call to sqrt has 2 args, want 1

Check’sargument, a set of Vars, accumulates the set ofvar iable names found wit hin the

expression. Eachofthese var iables mustbepresent inthe env iro nment for evaluation tosuc-

ce e d.Thisset islog ical lythe re sult of the cal l to Check,but because the method isrec ursive, it

is moreconvenientfor Check to popu lateaset passedasaparameter.The clientmustprovide

an empt y set in the initial cal l.

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

SECTION 7.9. EXAMPLE: EXPRESSION EVALUATOR203

In Sec tion 3.2, weplott eda function f(x,y) that was ﬁxe d at comp ile time. Now thatwecan

pars e, ch eck,and evaluate expressions instr ings, wecan bui ld aweb app lic ationthatreceives

an expressionatrun timefro m theclientand plots the sur face of thatfunction. Wecan use

the vars settoche ckthatthe expressionisafunctionofonlytwo var iables, x and y—thre e,

ac tually, since we’ll provide r,the radius,asaconvenience.And we’ll use the Check method to

rejec t ill-for med expressions beforeevaluation beg inssothatwedon’trep eat those che cks dur-

ingthe 40,000 evaluation s (100&100 cells, eachwit h four corners) ofthe functionthatfol low.

The parseAndCheck func tioncom binesthese parsingand che cking steps:

gopl.io/ch7/surface

import "gopl.io/ch7/eval"

func parseAndCheck(s string) (eval.Expr, error) {

if s == "" {

return nil, fmt.Errorf("empty expression")

}

expr, err := eval.Parse(s)

if err != nil {

return nil, err

}

vars := make(map[eval.Var]bool)

if err := expr.Check(vars); err != nil {

return nil, err

}

for v := range vars {

if v != "x" && v != "y" && v != "r" {

return nil, fmt.Errorf("undefined variable: %s", v)

}

return expr, nil

}

To makethisaweb app lic ation, allweneed isthe plot func tionbelow,whichhas the fami liar

sig natureofan http.HandlerFunc:

func plot(w http.ResponseWriter, r *http.Request) {

r.ParseForm()

expr, err := parseAndCheck(r.Form.Get("expr"))

if err != nil {

http.Error(w, "bad expr: "+err.Error(), http.StatusBadRequest)

return

}

w.Header().Set("Content-Type", "image/svg+xml")

surface(w, func(x, y float64) float64 {

r:=math.Hypot(x, y) // distance from (0,0)

return expr.Eval(eval.Env{"x": x, "y": y, "r": r})

})

}

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

204 CHAPTER 7. INTERFACES

Figure 7.7. Thesur faces of three functions:(a) sin(-x)*pow(1.5,-r);

(b) pow(2,sin(y))*pow(2,sin(x))/12;(c) sin(x*y/10)/10.

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

SECTION 7.10. TYPE ASSERTIONS 205

The plot func tionparsesand che cks the expressionspeciﬁed inthe HTTPrequestand usesit

to cre ate ananony mou s func tionoftwo var iables. Theanony mou s func tionhas the samesig-

nature asthe ﬁxe d func tion f from the originalsur face-plott ing program, but it evaluatesthe

us er-supp lie d expression. Theenv iro nment deﬁnes x, y,and the radius r.Final ly, plot calls

surface,whichisjustthe main func tionfro m gopl.io/ch3/surface,modiﬁed totakethe

func tiontoplotand the out put io.Writer as parameters, insteadofusingthe ﬁxe d func tion f

and os.Stdout.Figure7.7 shows three sur faces produce d by the program.

Exercis e 7.13: AddaString method to Expr to prett y-print the syntaxtre e.Che ckthatthe

resu lts, whenparsedagain, yield an equivalenttre e.

Exercis e 7.14: Deﬁne a newcon crete typ e that sat isﬁes the Expr interface andprovides a new

op erat ionsuchascomputing the minimum value ofits operands.Since the Parse func tion

do es notcre ate ins tances ofthisnew typ e,touse ityou will need tocon str uct a syntaxtre e

direc tly (or extendthe parser).

Exercis e 7.15: Wr ite a program thatreads a singleexpressionfro m thestand ard inp ut,

promptsthe usertoprovide values for any var iables, thenevaluatesthe expressioninthe

resu lting env iro nment.Handleall error s gracef ully.

Exercis e 7.16: Wr ite a web-b ased calc ulatorprogram.

7.10. Type Assertions

A ty peass ertion is anoperat ionapp lie d to anint erface value.Syntactic ally,itlooks like x.(T),

where x is anexpressionofanint erface typ e and T is a typ e,cal le d the ‘‘asserted’’ type.Atype

assertionche cks thatthe dynamic typ e of itsoperandmatch esthe ass erted typ e.

Thereare two possibi lit ies. First,ifthe ass erted typ e T is a con crete typ e,thenthe typ e asser-

tion che cks whether x’sdynamic typ e is ident icalto T.Ifthische cksucce e ds, the resultofthe

type ass ertionis x’sdynamic value,whose typ e is ofcours e T.Inother words,atyp e assertion

to a con crete typ e ext racts the con crete value fro m itsoperand. Ifthe che ckfai ls, thenthe

op erat ionpanics. For example:

var w io.Writer

w=os.Stdout

f:=w.(*os.File) // success: f == os.Stdout

c:=w.(*bytes.Buffer) // panic: interface holds *os.File, not *bytes.Buffer

Second,ifins teadthe ass erted typ e T is anint erface typ e,thenthe typ e assertionche cks

whet her x’sdynamic typ e sati sﬁes T.Ifthische cksucce e ds, the dynamic value isnot ext racte d;

theresultisstill anint erface value wit h thesametyp e andvalue comp onents, but the result

hasthe int erface typ e T.Inother words,atyp e assertiontoanint erface typ e ch angesthe typ e

of the expression, mak ingadif ferent(andusu allylarger) set ofmet hodsaccessible, but it

pres ervesthe dynamic typ e andvalue comp onentsinside the int erface value.

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

206 CHAPTER 7. INTERFACES

Af ter the ﬁrs t type ass ertionbelow,bot h w and rw hold os.Stdout so eachhas a dynamic typ e

of *os.File,but w,an io.Writer,exp oses onlythe ﬁle’s Write method,where as rw exp oses

its Read method too.

var w io.Writer

w=os.Stdout

rw := w.(io.ReadWriter) // success: *os.File has both Read and Write

w=new(ByteCounter)

rw = w.(io.ReadWriter) // panic: *ByteCounter has no Read method

No matterwhattyp e was asserted,ifthe operandisanil int erface value,the typ e assertion

fai ls. A type ass ertiontoalessrestr ictiveint erface typ e (one wit h fe wer met hods) israrely

ne e ded, as it beh avesjustlike an assig nment, except in the nil cas e.

w=rw//io.ReadWriter is assignable to io.Writer

w=rw.(io.Writer) // fails only if rw == nil

Of ten we’re not sureofthe dynamic typ e of anint erface value,and we’dliketotestwhether it

is som e partic ulartyp e.Ifthe typ e assertionapp earsinanassig nmentinwhichtwo results are

exp ected,suchasthe fol low ing declarat ions,the operat iondoesnot panic onfai lurebut

insteadretur ns an addition alsecon d resu lt, a boole an indic atingsuccess:

var w io.Writer = os.Stdout

f, ok := w.(*os.File) // success: ok, f == os.Stdout

b, ok := w.(*bytes.Buffer) // failure: !ok, b == nil

Thesecon d resu ltisconvent ion allyassig ned toavar iable named ok.Ifthe operat ionfai le d,

ok is false,and the ﬁrs t resu ltisequ altothe zerovalue ofthe ass erted typ e,whichinthis

exampleisanil *bytes.Buffer.

The ok resu ltisoften immediate lyusedtodecide whattodonext. Theextende d form ofthe

if st atement makes thisquite compact:

if f, ok := w.(*os.File); ok {

// ...use f...

}

Wh enthe operandofatyp e assertionisavar iable,rat her thaninv ent another namefor the

ne w lo cal variable, you’ll som etimessee the originalnamereused, shadowing the original, like

this:

if w, ok := w.(*os.File); ok {

// ...use w...

}

7.11. Discriminating Errorswith Type Assertions

Consider the set oferror s returned byﬁle operat ions inthe os package. I/O can failfor any

numb erofreasons,but three kinds offai lureoften mustbehandled dif ferently: ﬁle already

exists(forcre ate operat ions), ﬁle not found (forreadoperat ions), andper missiondenie d.The

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

SECTION 7.11. DISCRIMINATING ERRORS WITH TYPE ASSERTIONS 207

os packageprovides these three helperfunctions toclassif y thefai lureindic ated byagiven

error value:

package os

func IsExist(err error) bool

func IsNotExist(err error) bool

func IsPermission(err error) bool

Anaïve imp lementation of oneofthese pre dic ates might che ckthatthe error message con-

tainsacer tain subst ring,

func IsNotExist(err error) bool {

// NOTE: not robust!

return strings.Contains(err.Error(), "file does not exist")

}

butbecause the log ic forhandlingI/O erro rscan varyfro m on e pl atfor m to another,this

approach isnot robustand the samefai luremay berep orted wit h avar ietyofdif ferenterror

mess ages. Che cking for subst rings oferror messagesmay beusefuldur ingtesting toens ure

that functions fai l in the exp ected manner,but it’sinade quate for produc tioncode.

Amorereliableappro ach istorepresent str uctured error values usingadedic ated typ e.The

os packagedeﬁnesatyp e called PathError to des crib e fai lures invo l vinganoperat ionona

ﬁle pat h, li ke Open or Delete,and a var iantcal le d LinkError to des crib e fai lures ofopera-

tion s invo l vingtwo ﬁle pat hs, like Symlink and Rename.Here’s os.PathError:

package os

// PathError records an error and the operation and file path that caused it.

type PathError struct {

Op string

Path string

Err error

}

func (e *PathError) Error() string {

return e.Op + " " + e.Path + ": " + e.Err.Error()

}

Most clients areobliv iou s to PathError anddealwit h al l er ror s in a unifor m way bycal ling

their Error methods. Alt hough PathError’s Error method for msamessage bysimply con-

catenating the ﬁelds, PathError’sstr ucturepreserves the underly ing componentsofthe error.

Clients thatneed todistinguish one kindoffai lurefro m anot her can use a typ e assertionto

detec t thespeciﬁctyp e of the error ; thespeciﬁctyp e prov ides moredet ailthanasimplestr ing .

_, err := os.Open("/no/such/file")

fmt.Println(err) // "open /no/such/file: No such file or directory"

fmt.Printf("%#v\n", err)

// Output:

// &os.PathError{Op:"open", Path:"/no/such/file", Err:0x2}

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

208 CHAPTER 7. INTERFACES

That’show the three helperfunctions work.For example, IsNotExist,shown below,rep orts

whet her an erro r is equ alto syscall.ENOENT (§7.8) ortothe distinguishe d er ror

os.ErrNotExist (s ee io.EOF in §5.4.2), orisa*PathError whos e underly ing error isone of

thos e two.

import (

"errors"

"syscall"

)

var ErrNotExist = errors.New("file does not exist")

// IsNotExist returns a boolean indicating whether the error is known to

// report that a file or directory does not exist. It is satisfied by

// ErrNotExist as well as some syscall errors.

func IsNotExist(err error) bool {

if pe, ok := err.(*PathError); ok {

err = pe.Err

}

return err == syscall.ENOENT || err == ErrNotExist

}

Andhere it is in action:

_, err := os.Open("/no/such/file")

fmt.Println(os.IsNotExist(err)) // "true"

Of course, PathError’sstr uctureislostifthe error message iscom bine d into a largerstr ing ,

forins tance byacal l to fmt.Errorf.Error dis criminat ionmustusu allybedon e immediate ly

af ter the fai lingoperat ion, before an error ispro pagated tothe cal ler.

7.12. Querying Behaviorswith Int erfaceType Assertions

Thelog ic belowissimi lar tothe par t of the net/http we b server responsible for writing

HT TP he ader ﬁelds suchas "Content-type: text/html".The io.Writer w repres ents the

HT TP resp ons e;the bytes writt entoitare ult imate lysenttosom eon e’sweb brows er.

func writeHeader(w io.Writer, contentType string) error {

if _, err := w.Write([]byte("Content-Type: ")); err != nil {

return err

}

if _, err := w.Write([]byte(contentType)); err != nil {

return err

}

// ...

}

Becaus e the Write method requires a byteslice,and the value wewishtowrite isastr ing , a

[]byte(...) conv ersionisrequired.Thisconversionallocates memor y andmakes a copy,

butthe copyisthrow n away almostimmediate lyafter.Let’spretend thatthisisacorepar t of

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

SECTION 7.12. QUERYING BEHAVIORS WITH INTERFACE TYPE ASSERTIONS 209

theweb ser ver andthatour proﬁlinghas reveale d that thismemor y al location isslowing it

down.Can weavoid allo cat ingmemor y here?

The io.Writer interface tel lsusonlyone fac t ab out the con crete typ e that w holds: thatbytes

maybewritt entoit. Ifwelookbehindthe cur tainsofthe net/http package, wesee thatthe

dy namic typ e that w holdsinthisprogram also has a WriteString method thatallowsstr ings

to beefﬁcientlywritt entoit, avoidingthe need toallocateatemporar y copy.(Thismay seem

li keashotinthe dark, but a numberofimp ortanttyp es that sat isf y io.Writer also havea

WriteString method,including *bytes.Buffer, *os.File and *bufio.Writer.)

We c annot assume thatanarbit rar y io.Writer w also has the WriteString method.But we

candeﬁne a newint erface thathas justthismet hod and use a typ e assertiontotestwhether

thedynamic typ e of w satisﬁes thisnew int erface.

// writeString writes s to w.

// If w has a WriteString method, it is invoked instead of w.Write.

func writeString(w io.Writer, s string) (n int, err error) {

type stringWriter interface {

WriteString(string) (n int, err error)

}

if sw, ok := w.(stringWriter); ok {

return sw.WriteString(s) // avoid a copy

}

return w.Write([]byte(s)) // allocate temporary copy

}

func writeHeader(w io.Writer, contentType string) error {

if _, err := writeString(w, "Content-Type: "); err != nil {

return err

}

if _, err := writeString(w, contentType); err != nil {

return err

}

// ...

}

To avoid rep eat ingourselves, we’ve mov edthe che ckint o theutilit y func tion writeString,

butitissousefulthatthe stand ard librar y prov ides itas io.WriteString.Itisthe recom-

mended way towrite a str ing toan io.Writer.

Wh at’scur ious inthisexampleisthatthere isnostand ard int erface thatdeﬁnesthe

WriteString method and speciﬁes its required beh avior.Fur thermore, whether ornot a con-

cretetyp e satisﬁes the stringWriter interface isdeter mined onlybyits met hods, not byany

de clare d re lat ions hip bet weenitand the int erface typ e.Whatthismeans isthatthe technique

ab ove relies onthe assump tionthat if atyp e satisﬁes the int erface below, th en

WriteString(s) mu sthavethe sameeffec t as Write([]byte(s)).

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

210 CHAPTER 7. INTERFACES

interface {

io.Writer

WriteString(s string) (n int, err error)

}

Although io.WriteString do cuments its assump tion, few functions thatcal l it are likelyto

do cumentthatthe y to o make the sameassump tion. Deﬁningamet hod ofapar tic ulartyp e is

taken as an imp licitass ent for a cer tain beh avioral cont rac t. Ne wcomers to Go, esp eci ally

thos e from a backg round instron gly typ edlangu ages, may ﬁnd thislackofexplicitint ent ion

unsett ling, but it israrelyaproblem in prac tice.Wit h theexception of the emp tyint erface

interface{},int erface typ es areseldomsat isﬁed byunintended coincidence.

The writeString func tionabove usesatyp e assertiontosee whether a value ofageneral

interface typ e also sat isﬁes a morespeciﬁcint erface typ e,and ifso, itusesthe beh avior s of the

sp eciﬁcint erface.Thistechnique can beput togood use whether ornot the quer iedint erface

is stand ard like io.ReadWriter or user-deﬁne d li ke stringWriter.

It’sals o how fmt.Fprintf dist inguishesvalues thatsat isf y error or fmt.Stringer from all

ot her values. Wit hin fmt.Fprintf,there isa stepthatconvertsasingleoperandtoastr ing ,

somethinglikethis:

package fmt

func formatOneValue(x interface{}) string {

if err, ok := x.(error); ok {

return err.Error()

}

if str, ok := x.(Stringer); ok {

return str.String()

}

// ...all other types...

}

If x satisﬁes either ofthe two int erfaces, thatdeter mines the for matting ofthe value.Ifnot,the

defau ltcas e hand les allother typ es more orlessunifor mly usingreﬂec tion;we’ll ﬁnd out how

in Chapt er12.

Again, thismakes the assump tionthatany typ e with a String method sat isﬁes the beh avioral

cont rac t of fmt.Stringer,whichistoretur n astr ing suitablefor printing .

7.13. Type Switches

Interfaces areusedintwo distinc t st yles. Inthe ﬁrs t st yle,exemp liﬁed by io.Reader,

io.Writer, fmt.Stringer, sort.Interface, http.Handler,and error,anint erface’smet h-

odsexpress the simi lar ities ofthe con crete typ es that sat isf y theint erface but hide the rep-

resent ation det ails and int rinsic operat ions ofthose con crete typ es. Theemp hasisisonthe

methods, not onthe con crete typ es.

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

SECTION 7.13. TYPE SWITCHES 211

Thesecon d st yle exploitsthe abi lit y of anint erface value toholdvalues ofavar ietyofcon crete

typesand con sidersthe int erface tobethe union of those typ es. Type ass ertions are usedto

discriminateamong these typ es dy namic ally and tre ateachcas e dif ferently. Inthissty le, the

emph asisisonthe con crete typ es that sat isf y theint erface,not onthe int erface’smet hods(if

inde e d it has any), andthere isnohidingofinfor mat ion. We’ll des crib e interfaces usedthis

way as di scr imi nat eduni ons.

If you’re fami liar wit h objec t-oriente d prog ramming, you may recog nize thes e twosty les as

subtypepolymor phism and ad hocpolymor phism,but you needn’trememberthose ter ms. For

theremainder ofthischapt er, we’ll present examples ofthe secon d st yle.

Go’sAPI for quer yinganSQL dat abas e,likethose ofother langu ages, lets uscle anly sep arate

theﬁxe d part ofaquer y from the var iable par ts. An exampleclientmig htlooklikethis:

import "database/sql"

func listTracks(db sql.DB, artist string, minYear, maxYear int) {

result, err := db.Exec(

"SELECT * FROM tracks WHERE artist = ? AND ? <= year AND year <= ?",

artist, minYear, maxYear)

// ...

}

The Exec method replaces each '?' in the quer y st ringwit h an SQLlit eral denot ing the cor-

resp ondingargumentvalue,whichmay beaboole an, anumber, a str ing , or nil.Con str uct-

ingquer ies this way helps avoidSQL inj e ction att acks, in whichanadversary takes cont rol of

thequer y by exploit ing imp rop erquotation of inp utdat a. Wi thin Exec,wemig ht ﬁnd a func-

tion likethe one below,whichconvertseachargumentvalue toits literal SQLnot ation.

func sqlQuote(x interface{}) string {

if x == nil {

return "NULL"

}else if _, ok := x.(int); ok {

return fmt.Sprintf("%d", x)

}else if _, ok := x.(uint); ok {

return fmt.Sprintf("%d", x)

}else if b, ok := x.(bool); ok {

if b {

return "TRUE"

}

return "FALSE"

}else if s, ok := x.(string); ok {

return sqlQuoteString(s) // (not shown)

}else {

panic(fmt.Sprintf("unexpected type %T: %v", x, x))

}

A switch st atement simpliﬁes an if-else ch ain thatper for msaser ies of value equ ality tests.

An analogou s ty peswitch st atement simpliﬁes an if-else ch ain oftyp e assertions.

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

212 CHAPTER 7. INTERFACES

In its simplestfor m,atyp e sw itch looks like anordinar y sw itch statement inwhichthe oper-

andis x.(type)—that’slit eral lythe key word type—andeachcas e hasone ormoretyp es. A

type switch enables a multi-way branch bas edonthe int erface value’sdynamic typ e.The nil

case match esif x==nil,and the default case match esifnoother cas e do es. Atyp e sw itch

for sqlQuote wouldhavethese cas es:

switch x.(type) {

case nil: // ...

case int, uint: // ...

case bool: // ...

case string: // ...

default: // ...

}

As wit h an ordinar y sw itch statement (§1.8), cas es arecon sidered inorder and, whenamatch

is found,the cas e’sbodyisexe cut ed. Cas e order becom essig niﬁcant whenone ormorecas e

typesare int erfaces, since thenthere isapossibi lit y of two cas es matching. The position of the

default case rel ative tothe othersisimmater ial.No fallthrough is allow ed.

No tice thatinthe originalfunction, the log ic forthe bool and string casesneedsaccessto

thevalue extrac ted bythe typ e assertion. Since thisistypic al,the typ e sw itch statement has an

extende d form that binds the ext racte d value toanew var iable wit hin eachcas e:

switch x := x.(type) { /* ... */ }

Here we’ve cal le d thenew var iables x to o;aswit h type ass ertions,reuse ofvar iable names is

common.Likeaswitch st atement,atyp e sw itch imp licitlycre atesalexic al block, sothe dec-

larat ionofthe newvar iable cal le d x do es notconﬂic t with a var iable x in an out erblo ck. Each

case also imp licitlycre atesasep aratelexic al block.

Re writ ing sqlQuote to use the extende d form oftyp e sw itch makes itsig niﬁcant lycle arer :

func sqlQuote(x interface{}) string {

switch x := x.(type) {

case nil:

return "NULL"

case int, uint:

return fmt.Sprintf("%d", x) // x has type interface{} here.

case bool:

if x {

return "TRUE"

}

return "FALSE"

case string:

return sqlQuoteString(x) // (not shown)

default:

panic(fmt.Sprintf("unexpected type %T: %v", x, x))

}

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

SECTION 7.14. EXAMPLE: TOKEN-BASED XML DECODING 213

In thisversion, wit hin the blo ckofeachsingle-t ypecas e,the var iable x hasthe sametyp e as

thecas e.For ins tance, x hastyp e bool within the bool case and string within the string

case.Inall other cas es, x hasthe (interface) typ e of the switch op erand, whichis inter-

face{} in thisexample. Whenthe sameactionisrequired for multiplecas es, li ke int and

uint,the typ e sw itch makes iteasy tocom bine them.

Although sqlQuote accepts an argumentofany typ e,the functionrunstocomplet iononlyif

theargument’styp e match esone ofthe cas es in the typ e sw itch;other wis e it panics wit h an

‘‘unexp ected typ e’’ mess age . Although the typ e of x is interface{},wecon sider ita

di scr imi nat eduni on of int, uint, bool, string,and nil.

7.14. Example: To ken-Based XML Decoding

Section4.5 showe d howtodecodeJSONdocuments into Godat a st ruc tures wit h the Marshal

and Unmarshal func tions fro m the encoding/json package. The encoding/xml package

prov ides a similarAPI.Thisappro ach isconvenientwhenwewanttocon str uct a represen-

tation of the documenttre e,but that’sunnecessary for manyprograms. The encoding/xml

packageals o prov ides a lower-le vel tok en-bas ed APIfor decodingXML. Inthe token-b ased

st yle,the parsercon sumes the inp utand pro duces a streamoftokens, primarily offour

kind s—StartElement, EndElement, CharData,and Comment—each beingacon crete typ e in

the encoding/xml package. Eachcal l to (*xml.Decoder).Token returnsatoken.

Therelevantpar ts of the API are shown here:

encoding/xml

package xml

type Name struct {

Local string // e.g., "Title" or "id"

}

type Attr struct { // e.g., name="value"

Name Name

Value string

}

// A Token includes StartElement, EndElement, CharData,

// and Comment, plus a few esoteric types (not shown).

type Token interface{}

type StartElement struct { // e.g., <name>

Name Name

Attr []Attr

}

type EndElement struct { Name Name } // e.g., </name>

type CharData []byte // e.g., <p>CharData</p>

type Comment []byte // e.g.,

type Decoder struct{ /* ... */ }

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

214 CHAPTER 7. INTERFACES

func NewDecoder(io.Reader) *Decoder

func (*Decoder) Token() (Token, error) // returns next Token in sequence

The Token interface,whichhas nomet hods, isals o an exampleofadis criminated union. The

purpos e of a tradition alint erface like io.Reader is tohide detai lsofthe con crete typ es that

satisf y it sothatnew imp lementation s canbecre ate d;eachcon crete typ e is tre ate d unifor mly.

By contrast, the set ofcon crete typ es that sat isf y adis criminated unionisﬁxe d by the desig n

andexp osed,not hidden. Discriminated uniontyp es have few met hods; functions thatoper-

ateonthemare express edasa set ofcas es usingatyp e sw itch,wit h dif ferentlog ic in eachcas e.

The xmlselect prog ram belowext racts andprintsthe text found beneath cer tain elements in

an XML document tre e.Usingthe API above , it can do its job inasinglepassoverthe inp ut

withoutevermater ializingthe tre e.

gopl.io/ch7/xmlselect

// Xmlselect prints the text of selected elements of an XML document.

package main

import (

"encoding/xml"

"fmt"

"io"

"os"

"strings"

)

func main() {

dec := xml.NewDecoder(os.Stdin)

var stack []string // stack of element names

for {

tok, err := dec.Token()

if err == io.EOF {

break

}else if err != nil {

fmt.Fprintf(os.Stderr, "xmlselect: %v\n", err)

os.Exit(1)

}

switch tok := tok.(type) {

case xml.StartElement:

stack = append(stack, tok.Name.Local) // push

case xml.EndElement:

stack = stack[:len(stack)-1] // pop

case xml.CharData:

if containsAll(stack, os.Args[1:]) {

fmt.Printf("%s: %s\n", strings.Join(stack, " "), tok)

}

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

SECTION 7.14. EXAMPLE: TOKEN-BASED XML DECODING 215

// containsAll reports whether x contains the elements of y, in order.

func containsAll(x, y []string) bool {

for len(y) <= len(x) {

if len(y) == 0 {

return true

}

if x[0] == y[0] {

y=y[1:]

}

x=x[1:]

}

return false

}

Each timethe loopin main encounters a StartElement,itpushesthe element’snameont o a

st ack,and for each EndElement it popsthe namefro m thestack.The API guarante esthatthe

sequence of StartElement and EndElement tokens will bepro perly match ed, eveninill-

formed documents. Commentsare ignored.When xmlselect encounters a CharData,it

pr intsthe text onlyifthe stack containsall the elements named bythe command-lineargu-

ments, in order.

Thecommand below printsthe text ofany h2 elements appear ingbeneath two level s of div

elements. Its inputisthe XML speciﬁcation,its elf an XML document.

$gobuild gopl.io/ch1/fetch

$./fetch http://www.w3.org/TR/2006/REC-xml11-20060816 |

./xmlselect div div h2

html body div div h2: 1 Introduction

html body div div h2: 2 Documents

html body div div h2: 3 Logical Structures

html body div div h2: 4 Physical Structures

html body div div h2: 5 Conformance

html body div div h2: 6 Notation

html body div div h2: A References

html body div div h2: B Definitions for Character Normalization

...

Exercis e 7.17: Extend xmlselect so thatelements may beselec ted not justbyname, but by

their attr ibutestoo,inthe manner ofCSS, sothat, for ins tance,anelementlike

<div id="page" class="wide"> couldbeselec ted byamatching id or class as wel l as its

name.

Exercis e 7.18: Usingthe token-b ased decoder API, write a program thatwill readanarbit rar y

XML document and con str uct a tre e of gener ic no des thatrepresentsit. Nodes areoftwo

kind s: CharData no des represent text str ings, and Element no des represent named elements

andtheir attr ibutes. Eachelementnodehas a slice ofchi ld no des.

Yo u mayﬁnd the fol low ing declarat ions helpf ul.

import "encoding/xml"

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

216 CHAPTER 7. INTERFACES

type Node interface{} // CharData or *Element

type CharData string

type Element struct {

Type xml.Name

Attr []xml.Attr

Children []Node

}

7.15. A Fe w WordsofAdvice

Wh endesig ninganew package, nov ice Goprogrammersoften start bycre ating a set ofint er-

faces andonlylater deﬁne the con crete typ es that sat isf y them. Thisappro ach results in many

interfaces, eachofwhichhas onlyasingleimp lementation.Don’tdothat. Suchint erfaces are

unnecessary abstrac tions;the y also havearun-t imecost. You can restr ict whichmet hodsofa

type orﬁelds ofastr uct are visible outside a packageusingthe exp ort mechanism (§6.6).

Interfaces areonlyneeded whenthere are two ormorecon crete typ es that mustbedealt wit h

in a unifor m way.

We makeanexception tothisrulewhenanint erface issat isﬁed byasinglecon crete typ e but

that typ e cannot liveinthe samepackageasthe int erface because ofits dependencies. Inthat

case, an int erface isagood way todecoupletwo packages.

Becaus e interfaces areused in Go onlywhenthe y aresat isﬁed bytwo ormoretyp es, they

ne cessarily abstrac t away fro m thedet ails ofany par tic ularimp lementation.The resultis

smal ler interfaces wit h fe wer,simpler met hods, often justone aswit h io.Writer or

fmt.Stringer.Small int erfaces areeasier tosat isf y when new typ es come along . Agood rule

of thumb for int erface designis askonlyfor whatyou need.

Thiscon cludes our tourofmet hodsand int erfaces. Gohas gre atsup por t forthe obj e ct-

or iente d st yle ofprogramming, but thisdoesnot meanyou need touse itexc lusively. Not

ever ythingneed beanobj e ct; stand alone functions havetheir place,asdounenc apsulated

data typ es. Observethattoget her,the examples in the ﬁrs t ﬁve chapt ers ofthisbookcal l no

more thantwo dozen met hods, like input.Scan,asopp osedtoordinar y func tioncal lslike

fmt.Printf.

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

Goroutines and Channels

Conc urrentprogramming, the expressionofaprogram as a comp osition of several

autonomous activ ities, has never beenmoreimp ortantthanitistoday.Web ser vershandle

re quests for thous and s of clients at once. Tablet andphone appsrender animat ions inthe user

interface whi le simultane ously per for mingcomputation and networ k re quests in the back-

ground.Eventradition albatch pro blems—re adsom e data,compute, write som e output—us e

conc urrency tohide the latency ofI/O operat ions and toexploit a moder n comp uter’smany

processors,whichevery yeargrow in numberbut not inspeed.

Go enables two sty les ofcon cur rentprogramming. Thischapt erpresentsgoroutinesand

ch annel s,whichsup por t commu nicating sequent ial pro cesses or CSP,amodel ofcon cur rency

in whichvalues arepassedbet weenindep endentactiv ities (goroutines) but var iables arefor

themostpar t conﬁne d to a singleactiv ity.Chapt er9 cov ers som e aspectsofthe more tradi-

tion almodel of sh are d memory multithre ading,whichwill befami liar if you’ve usedthreads in

ot her mainst reamlangu ages. Chapt er9 als o points out som e importanthazards and pit fal lsof

conc urrentprogrammingthatwewon’tdelve int o in thischapt er.

Ev enthoug h Go’ssup por t forcon cur rency isone ofits gre atstrengt hs, reasoningabout con-

currentprogramsisinherentlyharder thanabout sequential ones, andint uit ion s acquired

from sequential programmingmay at times lead usast ray.Ifthisisyourﬁrs t encounterwit h

conc urrency,werecommend spendingalit tle extra timethin kingabout the examples in these

twochapt ers.

8.1. Goroutines

In Go, eachcon cur rentlyexe cut ing activ ity iscal le d a goro utine.Con sider a program thathas

twofunctions,one thatdoessom e comp utation and one thatwritessom e output,and assume

that neither functioncal lsthe other.Asequential program may cal l on e func tionand then

217

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

218 CHAPTER 8. GOROUTINES AND CHANNELS

call the other,but inaconcur rent prog ram with two ormoregoroutines, cal lsto both func tions

canbeactive at the sametime. We’ll see suchaprogram in a mom ent.

If you haveusedoperat ingsystemthreads orthreads inother langu ages, thenyou can assume

fornow thatagoroutine issimi lar toathread, and you’ll beabletowrite cor rec t prog rams.

Thedif ferences bet weenthreads and goroutinesare ess ent ial lyquant itative , notqualitative ,

andwill bedes crib ed in Sec tion 9.8.

Wh enaprogram starts, its onlygoroutine isthe one thatcal lsthe main func tion, sowecal l it

the main goroutine.New goroutinesare cre ate d by the go st atement.Syntactic ally,ago st ate-

ment isanordinar y func tionormet hod cal l preﬁxe d by the key word go.Ago st atement

caus esthe functiontobecal le d in a newly cre ate d goro utine.The go st atement its elf com-

pletes immediate ly:

f() // call f(); wait for it to return

go f() // create a new goroutine that calls f(); don'twait

In the examplebelow,the main goroutine computesthe 45t h Fibonacci number. Since ituses

theter r ibly inefﬁcientrec ursivealgor it hm, itrunsfor anappre ciabletime, dur ingwhichwe’d

li ketoprovide the userwit h avisualindic ationthatthe program isstill running, bydispl aying

an animated textual ‘‘spinner.’’

gopl.io/ch8/spinner

func main() {

go spinner(100 * time.Millisecond)

const n = 45

fibN := fib(n) // slow

fmt.Printf("\rFibonacci(%d) = %d\n", n, fibN)

}

func spinner(delay time.Duration) {

for {

for _, r := range `-\|/` {

fmt.Printf("\r%c", r)

time.Sleep(delay)

}

func fib(x int) int {

if x < 2 {

return x

}

return fib(x-1) + fib(x-2)

}

Af ter several secon dsofanimat ion, the fib(45) call retur nsand the main func tionprintsits

resu lt:

Fibonacci(45) = 1134903170

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

SECTION 8.2. EXAMPLE: CONCURRENT CLOCK SERVER 219

The main func tionthenretur ns. Whenthishappens,all goroutinesare abr uptlyter minated

andthe program exits. Other thanbyretur ningfro m main or exiting the program, there isno

prog rammat ic way for onegoroutine tostopanother,but aswewill see later,there are ways to

communic atewit h agoroutine torequestthatitstopits elf.

No tice how the program isexpress edasthe composition of two aut onom ous activ ities, spin-

ningand Fib onacci comp utation.Eachiswritt enasasep aratefunctionbut bot h make

prog resscon cur rently.

8.2. Example: ConcurrentClock Server

Networ kingisanatural domain in whichtouse con cur rency since ser verstypic ally handle

many conne ction s from their clients at once, eachclientbeingess ent ial lyindep endentofthe

ot hers. Inthissec tion,we’ll int roduce the net package, whichprovides the componentsfor

buildingnet wor ked clientand ser ver programsthatcommunic ateoverTCP,UDP,orUnix

domain sockets. The net/http packagewe’ve beenusingsince Chapt er1 isbui ltontop of

func tions fro m the net package.

Ourﬁrs t exampleisasequential clo ckser ver thatwritesthe cur renttimetothe clientonceper

second:

gopl.io/ch8/clock1

// Clock1 is a TCP server that periodically writes the time.

package main

import (

"io"

"log"

"net"

"time"

)

func main() {

listener, err := net.Listen("tcp", "localhost:8000")

if err != nil {

log.Fatal(err)

}

for {

conn, err := listener.Accept()

if err != nil {

log.Print(err) // e.g., connection aborted

continue

}

handleConn(conn) // handle one connection at a time

}

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

220 CHAPTER 8. GOROUTINES AND CHANNELS

func handleConn(c net.Conn) {

defer c.Close()

for {

_, err := io.WriteString(c, time.Now().Format("15:04:05\n"))

if err != nil {

return // e.g., client disconnected

}

time.Sleep(1 * time.Second)

}

The Listen func tioncre atesanet.Listener,anobj e ctthatlistens for incomingconne ction s

on a net wor k port,inthiscas e TCPpor t localhost:8000.The listener’s Accept method

blocks until anincomingconne ction requestismade,thenretur nsanet.Conn objec t rep-

resent ing the conne ction.

The handleConn func tionhandles one completeclientconne ction.Inaloop, itwritesthe cur-

rent time, time.Now(),tothe client. Since net.Conn satisﬁes the io.Writer interface,wecan

wr ite direc tly toit. Theloopend s when the write fai ls, mostlikelybecause the clienthas dis-

connec ted,atwhichpoint handleConn clos es itsside ofthe conne ction usingadefer red cal l to

Close andgoesbacktowaiting for another connec tion request.

The time.Time.Format method provides a way tofor mat dateand timeinfor mat ionby

example. Its argumentisatempl ate indic atinghow tofor mat a reference time, speciﬁcally

Mon Jan 2 03:04:05PM 2006 UTC-0700.The reference timehas eight components(dayofthe

we ek, month,day ofthe mont h,and soon). Any col lec tion of themcan appear in the Format

st ringinany order andinanumberoffor mats; the selec ted componentsofthe dateand time

wi l l be displ aye d in the selec ted for mats. Hereweare justusingthe hour,minut e , andsecon d

of the time. The time packagedeﬁnestempl atesfor manystand ard timefor mats, suchas

time.RFC1123.The samemechanism isusedinreverse whenparsingatimeusing

time.Parse.

To conne cttothe ser ver,we’ll need a clientprogram suchas nc (‘‘netc at’’), a stand ard utilit y

prog ram formanipu lat ingnet wor k connec tion s:

$gobuild gopl.io/ch8/clock1

$./clock1 &

$nclocalhost 8000

13:58:54

13:58:55

13:58:56

13:58:57

Theclientdispl ays the timesentbythe ser ver eachsecon d until weint err upt the clientwit h

Cont rol-C, whichonUnix systems isech oed as ^C by the shell. If nc or netcat is not ins tal le d

on yoursystem, you can use telnet or thissimpleGoversionof netcat that uses net.Dial

to conne cttoaTCP ser ver :

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

SECTION 8.2. EXAMPLE: CONCURRENT CLOCK SERVER 221

gopl.io/ch8/netcat1

// Netcat1 is a read-only TCP client.

package main

import (

"io"

"log"

"net"

"os"

)

func main() {

conn, err := net.Dial("tcp", "localhost:8000")

if err != nil {

log.Fatal(err)

}

defer conn.Close()

mustCopy(os.Stdout, conn)

}

func mustCopy(dst io.Writer, src io.Reader) {

if _, err := io.Copy(dst, src); err != nil {

log.Fatal(err)

}

Thisprogram reads dat a from the conne ction and writesittothe stand ard out put unt i l an

end-of-ﬁle condit ion or anerror occ urs. The mustCopy func tionisautilit y us edinseveral

examples in thissec tion.Let’srun two clients at the sametimeondif ferentter minals, one

shown tothe lef t andone tothe rig ht:

$gobuild gopl.io/ch8/netcat1

$./netcat1

13:58:54 $./netcat1

13:58:55

13:58:56

13:58:57

13:58:58

13:58:59

$killall clock1

The killall commandisaUnix utilit y that killsall pro cesseswit h thegiven name.

Thesecon d clientmustwaitunt i l theﬁrs t clientis ﬁnishe d becaus e theser ver is sequ ent ial;it

de alswit h on lyone clientatatime. Justone small change isneeded tomakethe ser ver con-

current: addingthe go ke yword tothe cal l to handleConn caus eseachcal l to run in its own

goro utine.

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

222 CHAPTER 8. GOROUTINES AND CHANNELS

gopl.io/ch8/clock2

for {

conn, err := listener.Accept()

if err != nil {

log.Print(err) // e.g., connection aborted

continue

}

go handleConn(conn) // handle connections concurrently

}

No w,multipleclients can receive the time at once:

$gobuild gopl.io/ch8/clock2

$./clock2 &

$gobuild gopl.io/ch8/netcat1

$./netcat1

14:02:54 $./netcat1

14:02:55 14:02:55

14:02:56 14:02:56

14:02:57 ^C

14:02:58

14:02:59 $./netcat1

14:03:00 14:03:00

14:03:01 14:03:01

^C 14:03:02

$killall clock2

Exercis e 8.1: Mo dif y clock2 to acceptapor t numb er, and write a program, clockwall,that

ac ts as a clientofseveral clo ckser versatonce, readingthe times fro m each one and displ aying

theresults in a table,akintothe wal l of clo cks seeninsom e bu sinessofﬁces. Ifyou have

accesstogeographic ally distr ibute d comp uters,run inst ances remot ely ; ot her wis e runlocal

inst ances ondif ferentpor tswit h faketimezon es.

$TZ=US/Eastern ./clock2 -port 8010 &

$TZ=Asia/Tokyo ./clock2 -port 8020 &

$TZ=Europe/London ./clock2 -port 8030 &

$clockwall NewYork=localhost:8010 London=localhost:8020 Tokyo=localhost:8030

Exercis e 8.2: Implementacon cur rentFileTransfer Pro tocol (FTP) ser ver.The ser ver should

interpretcommand s from eachclientsuchas cd to change direc tor y, ls to listadirec tor y, get

to sendthe contentsofaﬁle,and close to clos e theconne ction.You can use the stand ard ftp

command as the client, orwrite yourown.

8.3. Example: ConcurrentEcho Server

Theclo ckser ver usedone goroutine per connec tion.Inthissec tion,we’ll bui ld an ech o server

that usesmultiplegoroutinesper connec tion.Mostech o serversmerelywrite whate ver the y

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

SECTION 8.3. EXAMPLE: CONCURRENT ECHO SERVER 223

re ad, whichcan bedon e with this trivialversionof handleConn:

func handleConn(c net.Conn) {

io.Copy(c, c) // NOTE: ignoring errors

c.Close()

}

Amoreint erest ing ech o server might simulatethe reverb erations ofarealech o,wit h the

resp ons e loud at ﬁrs t ("HELLO!"), thenmoderate("Hello!")after a delay, thenquiet

("hello!")beforefadingtonot hing, as in thisversionof handleConn:

gopl.io/ch8/reverb1

func echo(c net.Conn, shout string, delay time.Duration) {

fmt.Fprintln(c, "\t", strings.ToUpper(shout))

time.Sleep(delay)

fmt.Fprintln(c, "\t", shout)

time.Sleep(delay)

fmt.Fprintln(c, "\t", strings.ToLower(shout))

}

func handleConn(c net.Conn) {

input := bufio.NewScanner(c)

for input.Scan() {

echo(c, input.Text(), 1*time.Second)

}

// NOTE: ignoring potential errors from input.Err()

c.Close()

}

We’ll need toupg rade ourclientprogram sothatitsends ter minal inputtothe ser ver whi le

also copying the ser ver respons e to the out put,whichpresentsanother opp ortunity touse

conc urrency :

gopl.io/ch8/netcat2

func main() {

conn, err := net.Dial("tcp", "localhost:8000")

if err != nil {

log.Fatal(err)

}

defer conn.Close()

go mustCopy(os.Stdout, conn)

mustCopy(conn, os.Stdin)

}

Whilethe main goroutine reads the stand ard inp utand sends ittothe ser ver,asecon d

goro utine reads and printsthe ser ver’srespons e.Whenthe main goroutine encount ers the

endofthe inp ut, for example, after the usertyp es Cont rol-D (^D)atthe ter minal (orthe

equivalentControl-Z onMicros oft Windows), the program stops, evenifthe other goroutine

st i l l haswor k to do. (We’ll see how tomakethe program waitfor bot h sides to ﬁnish once

we’ve int roduce d ch annel s in Sec tion 8.4.1.)

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

224 CHAPTER 8. GOROUTINES AND CHANNELS

In the sessionbelow,the client’sinp utislef t-aligne d andthe ser ver’srespons esare indente d.

Theclientshouts at the ech o server three times:

$gobuild gopl.io/ch8/reverb1

$./reverb1 &

$gobuild gopl.io/ch8/netcat2

$./netcat2

Hello?

HELLO?

Hello?

hello?

Is there anybody there?

IS THERE ANYBODY THERE?

Yooo-hooo!

Is there anybody there?

is there anybody there?

YOOO-HOOO!

Yooo-hooo!

yooo-hooo!

$killall reverb1

No tice thatthe thirdshout fro m theclientisnot dealt wit h until the secon d shout has petered

out, whichisnot ver y re alist ic. A re alech o wouldcon sistofthe comp ositi on of the three inde-

pendentshouts. Tosimulateit, we’ll need moregoroutines. Again, allweneed todoisadd

the go ke yword , this timetothe cal l to echo:

gopl.io/ch8/reverb2

func handleConn(c net.Conn) {

input := bufio.NewScanner(c)

for input.Scan() {

go echo(c, input.Text(), 1*time.Second)

}

// NOTE: ignoring potential errors from input.Err()

c.Close()

}

Thearguments tothe functionstarted by go areevaluate d when the go st atement its elf isexe-

cute d;thu s input.Text() is evaluate d in the main goroutine.

No w theech oes arecon cur rentand overl ap in time:

$gobuild gopl.io/ch8/reverb2

$./reverb2 &

$./netcat2

Is there anybody there?

IS THERE ANYBODY THERE?

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

SECTION 8.4. CHANNELS 225

Yooo-hooo!

Is there anybody there?

YOOO-HOOO!

is there anybody there?

Yooo-hooo!

yooo-hooo!

$killall reverb2

Al l that was required tomakethe ser ver use con cur rency,not justtohandleconne ction s from

mu ltipleclients but evenwit hin a singleconne ction,was the ins ertionoftwo go ke yword s.

Ho wever in addingthese key words,wehad tocon sider caref ullythatitissafetocal l methods

of net.Conn conc urrently, whichisnot truefor mosttyp es. We’ll dis cussthe cruci al concept

of concur rency saf ety in the next chapt er.

8.4. Channels

If goroutinesare the activ ities ofacon cur rentGoprogram, ch ann els arethe conne ction s

betweenthem. A ch annel isacommunic ationmechanism thatlets one goroutine sendvalues

to another goroutine.Eachchannel isacon duit for values ofapar tic ulartyp e,cal le d the

ch annel’s el ement type.The typ e of a channel whose elements havetyp e int is writt en

chan int.

To cre ate a channel,weuse the bui lt-in make func tion:

ch := make(chan int) // ch has type 'chan int'

As wit h maps, a channel isareference to the dat a st ruc turecre ate d by make.Whenwecopya

ch annel orpassone asanargumenttoafunction, weare copying a reference,socal ler and

callee refer tothe samedat a st ruc ture. Aswit h ot her reference typ es, thezerovalue ofachan-

nel is nil.

Tw o ch annel s of the sametyp e maybecompare d using ==.The comparisonistrueifbot h are

references tothe samechannel dat a st ruc ture. A ch annel may also becompare d to nil.

Achannel has two princip aloperat ions, send and re ceive,col lec tive lyknown as

commu nicati ons.Asend statement transmits a value fro m on e goro utine,through the chan-

nel,toanother goroutine exe cut ing a cor respondingreceive expression. Bot h op erat ions are

wr itt enusingthe <- op erator.Inasendstatement,the <- separates the channel and value op-

erands.Inareceive expression, <- precedes the channel operand. A re ceive expressionwhose

resu ltisnot usedisavalid statement.

ch <- x // a send statement

x=<-ch // a receive expression in an assignment statement

<-ch // areceive statement; result is discarded

Channel s supp ort a thirdoperat ion, cl ose,whichsets a ﬂag indic atingthatnomorevalues will

ever besentonthischannel;subsequentatt emp tstosendwill panic. Receive operat ions ona

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

226 CHAPTER 8. GOROUTINES AND CHANNELS

clos edchannel yield the values thathavebeensentunt i l no more values arelef t; anyreceive

op erat ions there after comp leteimmediate lyand yield the zerovalue ofthe channel’selement

type.

To clos e achannel,wecal l thebui lt-in close func tion:

close(ch)

Achannel cre ate d with a simplecal l to make is cal le d an unbuffered ch annel,but make accepts

an opt ion alsecon d argument, an integercal le d thechannel’s capacity.Ifthe cap acity isnon-

zero, make createsabuffered ch annel.

ch = make(chan int) // unbuffered channel

ch = make(chan int, 0) // unbuffered channel

ch = make(chan int, 3) // buffered channel with capacity 3

We’ll look at unbuf fered channel s ﬁrs t andbuf fered channel s in Sec tion 8.4.4.

8.4.1. Unbuffered Channels

Asendoperat iononanunbuf fered channel blo cks the sendinggoroutine unt i l anot her

goro utine exe cut esacorrespondingreceive onthe samechannel,atwhichpoint the value is

transmitt edand bot h goro utinesmay cont inue. Convers ely, if the receive operat ionwas

attemp ted ﬁrs t,the receiving goro utine isblo cke d until another goroutine per for msasendon

thesamechannel.

Communic ationoveranunbuf fered channel causesthe sendingand receiving goro utinesto

sy nchro niz e.Because ofthis, unbuffered channel s aresom etimescal le d sy nchro nou s ch annel s.

Wh enavalue issentonanunbuf fered channel,the receipt ofthe value happ ens before the

re awakeningofthe sendinggoroutine.

In dis cussions ofcon cur rency,whenwesay xhap pens beforey,wedon’tmeanmerelythat x

occurs earlier in timethan y;wemeanthatitisguarante e d to dosoand thatall its prior

ef fec ts, such as updates tovar iables, arecompleteand thatyou may relyonthem.

Wh en x neit her happens before y norafter y,wesay that xiscon cur rentwit h y.Thisdoesn’t

me anthat x and y arenecessarily simultane ous,merelythatwecannot assume anythingabout

their order ing. Aswe’ll see inthe next chapt er, it’snecessary toorder certain eventsdur ingthe

prog ram’sexe cut ion toavoid the pro blemsthatarise whentwo goro utinesaccessthe same

var iable conc urrently.

Theclientprogram in Sec tion 8.3 copies inputtothe ser ver in its main goroutine,sothe client

prog ram terminates as soonasthe inp utstreamclos es, even if the backg round goroutine is

st i l l work ing . To makethe program waitfor the backg round goroutine tocompletebefore

exiting , we use a channel tosynchro nize the two goro utines:

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

SECTION 8.4. CHANNELS 227

gopl.io/ch8/netcat3

func main() {

conn, err := net.Dial("tcp", "localhost:8000")

if err != nil {

log.Fatal(err)

}

done := make(chan struct{})

go func() {

io.Copy(os.Stdout, conn) // NOTE: ignoring errors

log.Println("done")

done <- struct{}{} // signal the main goroutine

}()

mustCopy(conn, os.Stdin)

conn.Close()

<-done // wait for background goroutine to finish

}

Wh enthe userclos es thestand ard inp utstream, mustCopy returnsand the main goroutine

calls conn.Close(),closingbot h halves ofthe networ k connec tion.Closingthe write half of

theconne ction causesthe ser ver tosee anend-of-ﬁle condit ion.Closingthe readhalf causes

thebackg round goroutine’scal l to io.Copy to retur n a ‘‘re adfro m clos edconne ction’’ er ror,

whichiswhy we’ve remov edthe error log ging; Exercis e 8.3 sug gests a bettersolut ion.(No tice

that the go st atement cal lsalit eral function, a common con str uction.)

Before itretur ns, the backg round goroutine logs a message , then sends a value onthe done

ch annel.The main goroutine waits until ithas receive d this value beforeretur ning. Asa

resu lt, the program always logs the "done" mess age beforeexiting .

Mess agessentoverchannel s have two imp ortantasp ects. Eachmessage has a value,but

sometimesthe fac t of communic ationand the moment atwhichitocc ursare justas

important. Wecal l mess ages events when wewishtostressthisasp ect.Whenthe event car-

ries noaddition alinfor mat ion, thatis, its solepur pos e is synchro nizat ion, we’ll emp hasize this

by usingachannel whose elementtyp e is struct{},thoug h it’scommontouse a channel of

bool or int forthe samepur pos e since done <- 1 is shorter than done <- struct{}{}.

Exercis e 8.3: In netcat3,the int erface value conn hasthe con crete typ e *net.TCPConn,which

repres ents a TCP connec tion.ATCPconne ction con sists oftwo halvesthatmay beclos ed

indep endentlyusingits CloseRead and CloseWrite methods. Modif y themain goroutine of

netcat3 to clos e on lythe write half ofthe conne ction sothatthe program will continueto

pr int the ﬁnalech oes fro m the reverb1 server evenafter the stand ard inp uthas beenclos ed.

(D oingthisfor the reverb2 server isharder ; seeExercis e 8.4.)

8.4.2. Pipelines

Channel s canbeusedtoconne ctgoroutinestoget her sothatthe out put of oneisthe inp utto

anot her.Thisiscal le d a pip elin e.The program below con sists ofthree goroutinesconne cte d

by two channel s, as shown sch ematically in Figure8.1.

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

228 CHAPTER 8. GOROUTINES AND CHANNELS

Figure 8.1. Athree-stage pip eline.

Theﬁrs t goro utine, counter,generates the int egers 0,1,2,..., andsends themoverachannel to

thesecon d goro utine, squarer,whichreceiveseachvalue,squ aresit, andsends the resultover

anot her channel tothe thirdgoroutine, pr int er,whichreceivesthe squ are d values andprints

them. For clarity ofthisexample, wehaveint ent ion allychosenver y simplefunctions,thoug h

of cours e they are too computation allytrivialtowar ranttheir own goroutinesinarealist ic

prog ram.

gopl.io/ch8/pipeline1

func main() {

naturals := make(chan int)

squares := make(chan int)

// Counter

go func() {

for x := 0; ; x++ {

naturals <- x

}

}()

// Squarer

go func() {

for {

x:=<-naturals

squares <- x * x

}

}()

// Printer (in main goroutine)

for {

fmt.Println(<-squares)

}

As you mig htexp ect,the program printsthe inﬁnite ser ies of squ ares0,1,4,9,and soon.

Pipelines like thismay befound inlon g-r unningser ver programswhere channel s areusedfor

lifelon g communic ationbet weengoroutinescontaininginﬁnite loops. But what if we wantto

send onlyaﬁnite numberofvalues through the pip eline?

If the sender knows thatnofur thervalues will everbesentonachannel,itisusefultocom-

munic atethisfac t to the receivergoroutinessothatthe y canstopwaiting . Thisisaccom-

plishe d by cl osing thechannel usingthe bui lt-in close func tion:

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

SECTION 8.4. CHANNELS 229

close(naturals)

Af ter a channel has beenclos ed, any fur thersendoperat ions onitwill panic. After the clos ed

ch annel has been drai ned,thatis, afterthe lastsentelementhas beenreceive d,all subsequent

re ceive operat ions will pro ceed wit houtblo cking but will yield a zerovalue.Closingthe natu-

rals ch annel above wouldcause the squ arer’slooptospinasitreceivesanever-endingstream

of zerovalues, andtosendthese zeros tothe print er.

Thereisnoway totestdirec tly whether a channel has beenclos ed, but there isavar iantofthe

re ceive operat ionthatpro duces two results: the receive d ch annel element, plu s aboole an

value,convent ion allycal le d ok,whichis true forasuccessf ulreceive and false forareceive

on a clos edand drained channel.Usingthisfeature , we can modif y thesqu arer’slooptostop

when the naturals ch annel isdrained and clos e the squares ch annel intur n.

// Squarer

go func() {

for {

x, ok := <-naturals

if !ok {

break // channel was closed and drained

}

squares <- x * x

}

close(squares)

}()

Becaus e thesyntaxabove isclumsyand thispattern iscommon, the langu agelets ususe a

range lo optoiterateoverchannel s to o.Thisisamoreconvenientsyntaxfor receiving all the

values sentonachannel and ter minat ingthe loopafter the lastone.

In the pip elinebelow,whenthe count ergoroutine ﬁnishesits loop after 100 elements, itclos es

the naturals ch annel,causingthe squ arerto ﬁnish its loop and clos e the squares ch annel.

(Inamorecomplex program, itmig htmakesense for the count erand squ arerfunctions to

defer the cal lsto close at the outset.) Final ly, the main goroutine ﬁnishesits loop and the

prog ram exits.

gopl.io/ch8/pipeline2

func main() {

naturals := make(chan int)

squares := make(chan int)

// Counter

go func() {

for x := 0; x < 100; x++ {

naturals <- x

}

close(naturals)

}()

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

230 CHAPTER 8. GOROUTINES AND CHANNELS

// Squarer

go func() {

for x := range naturals {

squares <- x * x

}

close(squares)

}()

// Printer (in main goroutine)

for x := range squares {

fmt.Println(x)

}

Yo u ne e dn’tclos e ever y ch annel whenyou’ve ﬁnishe d with it. It’sonlynecessary toclos e a

ch annel whenitisimp ortanttotel l thereceiving goro utinesthatall dat a have beensent. A

ch annel thatthe garb agecol lec tor deter mines tobeunreach ablewill haveits res ources

re claimed whether ornot itisclos ed. (Don’tconfuse thiswit h theclos e op erat ionfor open

ﬁles. It is importanttocal l the Close method onevery ﬁle whenyou’ve ﬁnishe d with it.)

At tempt ing toclos e an already-clos edchannel causesapanic, as does closinganil channel.

Closingchannel s hasanother use asabro adc astmechanism, whichwe’ll cov er in Sec tion 8.9.

8.4.3. Unidirectional Channel Types

As programsgrow, itisnatural tobre akuplarge functions int o smal ler pieces. Our pre vious

exampleusedthree goroutines, communic atingovertwo channel s,whichwerelocal variables

of main.The program natural lydiv ides into three functions:

func counter(out chan int)

func squarer(out, in chan int)

func printer(in chan int)

The squarer func tion, sitt ing inthe midd le of the pip eline, takes two parameters, the inp ut

ch annel and the out put channel.Bot h have the sametyp e,but their intended usesare

opp osite: in is onlytobereceive d from,and out is onlytobesentto. The names in and out

conv eythisint ent ion,but still,not hingpre vents squarer from sendingto in or receiving

from out.

Thisarrangement istypic al.Whenachannel issup plie d as a functionparameter,itisnearly

always wit h theint ent thatitbeusedexc lusivelyfor sendingorexc lusivelyfor receiving .

To documentthisint ent and pre ventmisus e,the Gotyp e systemprovides unidirec tional ch an-

nel typ es that expos e on lyone orthe other ofthe sendand receive operat ions.The typ e

chan<- int,asend-only ch annel of int,allowssends but not receives. Convers ely,the typ e

<-chan int,are ceive-only ch annel of int,allowsreceivesbut not sends.(Theposition of the

<- ar row rel ative tothe chan ke yword isamnemonic.) Violation s of thisdis cip lineare

detec ted atcompi letime.

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

SECTION 8.4. CHANNELS 231

Sincethe close op erat ionass ertsthatnomoresends will occ ur on a channel,onlythe send-

inggoroutine isinaposition tocal l it,and for thisreasonitisacompi le-t imeerror toatt emp t

to clos e areceive-onlychannel.

Here’sthe squ aring pip elineoncemore, thistimewit h unidirec tion alchannel typ es:

gopl.io/ch8/pipeline3

func counter(out chan<- int) {

for x := 0; x < 100; x++ {

out <- x

}

close(out)

}

func squarer(out chan<- int, in <-chan int) {

for v := range in {

out <- v * v

}

close(out)

}

func printer(in <-chan int) {

for v := range in {

fmt.Println(v)

}

func main() {

naturals := make(chan int)

squares := make(chan int)

go counter(naturals)

go squarer(squares, naturals)

printer(squares)

}

Thecal l counter(naturals) implicitlyconverts naturals,a value oftyp e chan int,tothe

type ofthe parameter, chan<- int.The printer(squares) call doesasimi lar implicitcon-

versionto <-chan int.Conversions fro m bidirec tion altounidirec tion alchannel typ es are

permit ted inany assig nment. Thereisnogoing back, how ever: onceyou haveavalue ofa

unidirec tion altyp e such as chan<- int,there isnoway toobt ain fro m it a value oftyp e

chan int that referstothe samechannel dat a st ruc ture.

8.4.4. Buffered Channels

Abuf fered channel has a queue ofelements. Thequeue’smaximum size isdeter mined whenit

is cre ate d, by the cap acity argumentto make.The statement below cre atesabuf fered channel

capableofholdingthree string values. Figure8.2 isagraphic al repres entation of ch andthe

ch annel towhichitrefers.

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

232 CHAPTER 8. GOROUTINES AND CHANNELS

ch = make(chan string, 3)

Figure 8.2. An emp tybuf fered channel.

Asendoperat iononabuf fered channel ins ertsanelementatthe backofthe queue,and a

re ceive operat ionremov es an elementfro m thefro nt. Ifthe channel isfull,the sendoperat ion

blocks its goroutine unt i l sp ace ismade availablebyanother goroutine’sreceive . Conv ers ely,if

thechannel isemp ty, a receive operat ionblo cks until a value issentbyanother goroutine.

We can senduptothree values onthischannel wit houtthe goroutine blo cking:

ch <- "A"

ch <- "B"

ch <- "C"

At thispoint,the channel isfull (Figure8.3), andafourthsendstatement wou ldblo ck.

Figure 8.3. Afull buf fered channel.

If wereceive one value,

fmt.Println(<-ch) // "A"

thechannel isneither full nor emp ty(Figure8.4), soeit her a sendoperat ionorareceive oper-

at ioncou ldpro ceed wit houtblo cking . In thisway,the channel’sbuf fer decoup les the sending

andreceiving goro utines.

Figure 8.4. Apar tiallyfull buf fered channel.

In the unlikelyevent thataprogram needstoknowthe channel’sbuf fer cap acity,itcan be

obtained bycal lingthe bui lt-in cap func tion:

fmt.Println(cap(ch)) // "3"

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

SECTION 8.4. CHANNELS 233

Wh enapp lie d to a channel,the bui lt-in len func tionretur nsthe numberofelements cur-

rent lybuf fered.Since in a conc urrentprogram thisinfor mat ionislikelytobestale as soonas

it isret rie ved,its value islimite d, butitcou ldcon ceivably beusefuldur ingfau ltdiagnosisor

perfor mance opt imizat ion.

fmt.Println(len(ch)) // "2"

Af ter two morereceive operat ions the channel isemp tyagain, andafourthwou ldblo ck:

fmt.Println(<-ch) // "B"

fmt.Println(<-ch) // "C"

In thisexample, the sendand receive operat ions were all per for med bythe samegoroutine,

butinrealprogramsthe y areusu allyexe cut edbydif ferentgoroutines. Nov ices aresom etimes

temp ted touse buf fered channel s within a singlegoroutine asaqueue,lured bytheir ple as-

inglysimplesyntax, but thisisamistake . Channel s aredeeply conne cte d to goroutine

sche duling, and wit houtanother goroutine receiving fro m thechannel,asender—andperhaps

thewhole program—risksbecomingblo cke d fore ver.Ifall you need isasimplequeue,make

on e usingaslice.

Theexamplebelow shows an applic ationofabuf fered channel.Itmakes paral lelrequests to

thre e mi rro rs,thatis, equivalentbut geographic ally distr ibute d servers. Itsends their

resp ons esoverabuf fered channel,thenreceivesand retur nsonlythe ﬁrs t resp ons e,whichis

thequickest one toarrive . Thus mirroredQuery returnsaresultevenbeforethe two slower

servershaveresponded.(In cidentally, it’squite nor mal for several goroutinestosendvalues to

thesamechannel con cur rently, as in thisexample, ortoreceive fro m thesamechannel.)

func mirroredQuery() string {

responses := make(chan string, 3)

go func() { responses <- request("asia.gopl.io") }()

go func() { responses <- request("europe.gopl.io") }()

go func() { responses <- request("americas.gopl.io") }()

return <-responses // return the quickest response

}

func request(hostname string) (response string) { /* ... */ }

Hadweusedanunbuf fered channel,the two slowergoroutineswou ldhavegot ten stuck trying

to sendtheir respons esonachannel fro m whichnogoroutine will everreceive . Thissit-

uation,cal le d a goro utine leak,wou ldbeabug . Un likegarb agevar iables, leaked goroutinesare

notaut omat ical lycol lec ted,soitisimp ortanttomakesurethatgoroutinester minatethem-

selves whennolon g erneeded.

Thechoice bet weenunbuf fered and buf fered channel s,and the choice ofabuf fered channel’s

capacity,may bot h af fec t thecor rec tnessofaprogram. Unbuf fered channel s give stron g er

sy nchro nizat ionguarante esbecause ever y send operat ionissynchro nize d with its cor-

resp ondingreceive;wit h buffered channel s,these operat ions are decoupled.Als o,whenwe

know anupp erbound onthe numberofvalues thatwill besentonachannel,it’snot unu sual

to cre ate a buf fered channel ofthatsize andper for m al l thesends beforethe ﬁrs t value is

re ceive d.Fai luretoallocatesufﬁcientbuf fer cap acity wou ldcause the program todeadlock.

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

234 CHAPTER 8. GOROUTINES AND CHANNELS

Channel buf fer ingmay also affec t prog ram perfor mance.Imagine three cooks in a cakeshop,

on e baking , on e icing, and one ins cribing eachcakebeforepassingitontothe next cook inthe

assemb l y line. Inakit c hen wit h litt lespace,eachcookthathas ﬁnishe d acakemustwaitfor

thenext cook tobecom e re ady toacceptit; thisrende zvou s is analogou s to communic ation

ov er an unbuf fered channel.

If there isspace for onecakebet weeneachcook, a cook may place a ﬁnishe d cake there and

immediate lystart wor k on the next; thisisanalogou s to a buf fered channel wit h capacity 1.So

long asthe cooks wor k at aboutthe samerateonaverage, mostofthese handovers pro ceed

quickly, smo othingout transientdif ferences in their respectiverates. Morespace bet ween

co oks—largerbuf fers—cansmo oth out bigger transientvar iat ions intheir rates wit hout

st allingthe ass emb l y line, suchashappens whenone cooktakes a short bre ak, thenlater

rushestocatch up.

On the other hand, ifanearlier stage ofthe ass emb l y lineiscon sistent lyfasterthanthe fol low-

ingstage , thebuf fer bet weenthemwill spend most ofits timefull.Convers ely,ifthe later

st age isfaster, the buf fer will usu allybeemp ty. A buffer provides nobeneﬁt in thiscas e.

Theass emb l y linemet aphorisausefulone for channel s andgoroutines. For example, ifthe

second stage ismoreelaborate, a singlecookmay not beabletokeepupwit h thesup ply fro m

theﬁrs t co okormeetthe demandfro m thethird. Tosolvethe pro blem, wecou ldhireanother

co oktohelpthe secon d,per for mingthe sametaskbut wor kingindep endently. Thisisanalo-

gous tocre ating another goroutine communic atingoverthe samechannel s.

We don’thavespace toshowithere, but the gopl.io/ch8/cake packagesimulates thiscake

shop, wit h several parametersyou can vary. Itincludes benchmarks(§11.4) for a few ofthe

scenar ios des crib edabove .

8.5. Looping in Parallel

In thissec tion,we’ll explore som e common con cur rency patternsfor exe cut ing all the itera-

tion s of a loopinparal lel. We’ll con sider the pro blem ofpro ducingthumbnail-size images

from a set offull-size ones. The gopl.io/ch8/thumbnail packageprovides an ImageFile

func tionthatcan scale a singleimage . We won’tshowits implementation but it can be

down loade d from gopl.io.

gopl.io/ch8/thumbnail

package thumbnail

// ImageFile reads an image from infile and writes

// a thumbnail-size version of it in the same directory.

// It returns the generated file name, e.g., "foo.thumb.jpg".

func ImageFile(infile string) (string, error)

Theprogram below loops overalistofimage ﬁle names andpro duces a thumbnail for each

on e:

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

SECTION 8.5. LOOPING IN PARALLEL 235

gopl.io/ch8/thumbnail

// makeThumbnails makes thumbnails of the specified files.

func makeThumbnails(filenames []string) {

for _, f := range filenames {

if _, err := thumbnail.ImageFile(f); err != nil {

log.Println(err)

}

Obviou sly the order in whichwepro cessthe ﬁles doesn’tmatter, since eachscalingoperat ion

is indep endentofall the others. Pro blemslikethisthatcon sistent ire lyofsubproblemsthat

arecompletelyindep endentofeachother aredes crib edas embarrassingly paral lel.Emb arrass-

inglyparal lelpro blemsare the easiestkindtoimp lementcon cur rentlyand enj oyper for mance

that scales linearlywit h theamountofparal lelism.

Let’sexe cut e al l thes e op erat ions inparal lel, there byhidingthe latency ofthe ﬁle I/O and

usingmultipleCPUsfor the image-s calingcomputation s.Our ﬁrs t attemp t at a conc urrent

versionjustaddsago ke yword . We’ll ignoreerror s fornow and addressthemlater.

// NOTE: incorrect!

func makeThumbnails2(filenames []string) {

for _, f := range filenames {

go thumbnail.ImageFile(f) // NOTE: ignoring errors

}

Thisversionrunsreallyfast—to o fast, in fact, since ittakes lesstimethanthe original, even

when the slice of ﬁle names cont ainsonlyasingleelement. Ifthere’snoparal lelism, how can

thecon cur rentversionpossibly run faster? Theansweristhat makeThumbnails returns

before ithas ﬁnishe d doingwhatitwas sup pos edtodo. Itstartsall the goroutines, one per ﬁle

name,but doesn’twaitfor themto ﬁnish.

Thereisnodirec t way towaitunt i l agoroutine has ﬁnishe d,but wecan change the inner

goro utine torep ort its comp let iontothe out ergoroutine bysendinganevent onashare d

ch annel.Since weknowthatthere are exac tly len(filenames) inner goroutines, the out er

goro utine need onlycount thatmanyeventsbeforeitretur ns:

// makeThumbnails3 makes thumbnails of the specified files in parallel.

func makeThumbnails3(filenames []string) {

ch := make(chan struct{})

for _, f := range filenames {

go func(f string) {

thumbnail.ImageFile(f) // NOTE: ignoring errors

ch <- struct{}{}

}(f)

}

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

236 CHAPTER 8. GOROUTINES AND CHANNELS

// Wait for goroutines to complete.

for range filenames {

<-ch

}

No tice thatwepassedthe value of f as an explicitargumenttothe lit eral functionins teadof

usingthe declarat ionof f from the enclosing for lo op:

for _, f := range filenames {

go func() {

thumbnail.ImageFile(f) // NOTE: incorrect!

// ...

}()

}

Re call the pro blem ofloopvar iable capture inside an anonymou s func tion, descr ibedin

Section5.6.1. Above , thesinglevar iable f is share d by all the anony mou s func tionvalues and

up dated bysuccessive loopiterat ions.Bythe timethe newgoroutinesstart exe cut ing the lit-

eral function, the for lo opmay haveupdated f andstarted another iterat ionor(more likely)

ﬁnishe d entire ly, sowhenthese goroutinesreadthe value of f,the y al l observeittohavethe

value ofthe ﬁnalelementofthe slice.Byaddinganexplicitparameter,weens ure thatweuse

thevalue of f that iscur rentwhenthe go st atement isexe cut ed.

Wh atifwewanttoretur n values fro m each wor ker goroutine tothe main one?Ifthe cal l to

thumbnail.ImageFile fai lstocre ate a ﬁle,itretur ns an error.The next versionof

makeThumbnails returnsthe ﬁrs t er ror it receivesfro m anyofthe scalingoperat ions:

// makeThumbnails4 makes thumbnails for the specified files in parallel.

// It returns an error if any step failed.

func makeThumbnails4(filenames []string) error {

errors := make(chan error)

for _, f := range filenames {

go func(f string) {

_, err := thumbnail.ImageFile(f)

errors <- err

}(f)

}

for range filenames {

if err := <-errors; err != nil {

return err // NOTE: incorrect: goroutine leak!

}

return nil

}

Thisfunctionhas a subtlebug . Wh enitencount ers the ﬁrs t non-ni l er ror,itretur nsthe error

to the cal ler,leaving nogoroutine drainingthe errors ch annel.Eachremainingwor ker

goro utine will blo ckforever whenittries tosendavalue onthatchannel,and will never

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

SECTION 8.5. LOOPING IN PARALLEL 237

terminate. Thissit uat ion, a goroutine leak(§8.4.4), may cause the whole program toget stuck

or torun out of memor y.

Thesimplestsolut ion istouse a buf fered channel wit h sufﬁcientcap acity thatnowor ker

goro utine will blo ckwhenitsends a message . (Analt ernat ive solut ion istocre ate another

goro utine todrain the channel whi le themain goroutine retur nsthe ﬁrs t er ror wit houtdel ay.)

Thenext versionof makeThumbnails us esabuf fered channel toretur n thenames ofthe gen-

erated image ﬁles along wit h anyerror s.

// makeThumbnails5 makes thumbnails for the specified files in parallel.

// It returns the generated file names in an arbitrary order,

// or an error if any step failed.

func makeThumbnails5(filenames []string) (thumbfiles []string, err error) {

type item struct {

thumbfile string

err error

}

ch := make(chan item, len(filenames))

for _, f := range filenames {

go func(f string) {

var it item

it.thumbfile, it.err = thumbnail.ImageFile(f)

ch <- it

}(f)

}

for range filenames {

it := <-ch

if it.err != nil {

return nil, it.err

}

thumbfiles = append(thumbfiles, it.thumbfile)

}

return thumbfiles, nil

}

Ourﬁnalversionof makeThumbnails,below,retur nsthe tot alnumberofbytes occ upied by

thenew ﬁles. Unlikethe pre vious versions,how ever, itreceivesthe ﬁle names not asaslice but

ov erachannel ofstr ings, sowecannot pre dic t thenumberofloopiterat ions.

To k nowwhenthe lastgoroutine has ﬁnishe d (w hichmay not bethe lastone tostart), weneed

to increment a count erbeforeeachgoroutine startsand decrement itaseachgoroutine ﬁn-

ishes. Thisdemands a speci al kind ofcount er, one thatcan besafelymanipu lated fro m

mu ltiplegoroutinesand thatprovides a way towaitunt i l it becom eszero. Thiscount ertyp e is

know n as sync.WaitGroup,and the codebelow shows how touse it:

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

238 CHAPTER 8. GOROUTINES AND CHANNELS

// makeThumbnails6 makes thumbnails for each file received from the channel.

// It returns the number of bytes occupied by the files it creates.

func makeThumbnails6(filenames <-chan string) int64 {

sizes := make(chan int64)

var wg sync.WaitGroup // number of working goroutines

for f := range filenames {

wg.Add(1)

// worker

go func(f string) {

defer wg.Done()

thumb, err := thumbnail.ImageFile(f)

if err != nil {

log.Println(err)

return

}

info, _ := os.Stat(thumb) // OK to ignore error

sizes <- info.Size()

}(f)

}

// closer

go func() {

wg.Wait()

close(sizes)

}()

var total int64

for size := range sizes {

total += size

}

return total

}

No tethe asy mmet ryinthe Add and Done methods. Add,whichincrementsthe count er, must

be cal le d before the wor ker goroutine starts, not wit hin it; other wis e we wouldnot besurethat

the Add happ ens before the ‘‘clos er’’ goro utine cal ls Wait.Als o, Add takesaparameter,but Done

do es not; it’sequivalentto Add(-1).Weuse defer to ens ure thatthe count erisdecrement ed

even in the error cas e.The str uctureofthe codeabove isacommonand idiomaticpattern for

lo oping inparal lelwhenwedon’tknowthe numberofiterat ions.

The sizes ch annel car r ies each ﬁle size backtothe main goroutine,whichreceivesthem

usingarange lo opand computesthe sum. Obs erve how wecre ate a clos er goro utine that

waits for the wor kersto ﬁnish beforeclosingthe sizes ch annel.These two operat ions,wait

andclos e,mustbecon cur rentwit h theloopover sizes.Con sider the alt ernat ives: if the wait

op erat ionwereplace d in the main goroutine beforethe loop, itwou ldnever end, and ifplace d

af ter the loop, itwou ldbeunreach ablesince wit h nothingclosingthe channel,the loopwou ld

ne ver ter minate.

Figure8.5 illustrates the sequence ofeventsinthe makeThumbnails6 func tion. Thever tic al

lines represent goroutines. Thethin seg ments indic atesle ep,the thickseg ments activity.The

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

SECTION 8.6. EXAMPLE: CONCURRENT WEB CRAWLER 239

Figure 8.5. Thesequence ofeventsin makeThumbnails6.

di agon alarrowsindic ateeventsthatsynchro nize one goroutine wit h anot her.Time ﬂows

down.Not ice how the main goroutine spend s most ofits timeinthe range lo opasleep, wait-

ingfor a wor ker tosendavalue orthe clos er to clos e thechannel.

Exercis e 8.4: Mo dif y the reverb2 server touse a sync.WaitGroup perconne ction tocount

thenumberofactive echo goro utines. Whenitfal lstozero, clos e thewrite half ofthe TCP

connec tion asdes crib edinExercis e 8.3. Ver ifythatyourmodiﬁed netcat3 clientfro m that

exercise waits for the ﬁnalech oes ofmultiplecon cur rentshouts, evenafter the stand ard inp ut

hasbeenclos ed.

Exercis e 8.5: Take anexist ing CPU-b oundsequential program, suchasthe Mandelbro t

prog ram of Sec tion 3.3 orthe 3-D sur face comp utation of Sec tion 3.2, andexe cut e itsmain

lo op in paral lelusingchannel s forcommunic ation. How muchfasterdoesitrun ona

mu ltiprocessormachine? Whatisthe opt imalnumberofgoroutinestouse?

8.6. Example: ConcurrentWeb Crawler

In Sec tion 5.6, wemade a simpleweb craw ler that explore d thelin k graphofthe web in

breadt h-ﬁrs t order.Inthissec tion,we’ ll makeitcon cur rentsothatindep endentcal lsto crawl

canexploit the I/O paral lelism availableinthe web.The crawl func tionremainsexac tly asit

was in gopl.io/ch5/findlinks3:

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

240 CHAPTER 8. GOROUTINES AND CHANNELS

gopl.io/ch8/crawl1

func crawl(url string) []string {

fmt.Println(url)

list, err := links.Extract(url)

if err != nil {

log.Print(err)

}

return list

}

Themain functionres embles breadthFirst (§5.6). Asbefore, a wor klist records the queue of

it ems thatneed pro cessing, eachitembeingalistofURLs tocraw l,but thistime, ins teadof

repres enting the queue usingaslice,weuse a channel.Eachcal l to crawl occurs inits own

goro utine and sends the lin ks it dis cov ers backtothe wor klist.

func main() {

worklist := make(chan []string)

// Start with the command-line arguments.

go func() { worklist <- os.Args[1:] }()

// Crawl the web concurrently.

seen := make(map[string]bool)

for list := range worklist {

for _, link := range list {

if !seen[link] {

seen[link] = true

go func(link string) {

worklist <- crawl(link)

}(link)

}

No tice thatthe craw l goro utine takes link as an explicitparameter toavoid the pro blem of

lo opvar iable capture weﬁrs t saw in Sec tion 5.6.1. Als o notice thatthe initial sendofthe com-

mand-linearguments tothe wor klist mustrun in its own goroutine toavoid dead lock,astuck

situ ation inwhichbot h themain goroutine and a craw ler goro utine att emp t to sendtoeach

ot her whi le neit her isreceiving . An alt ernat ive solut ion wou ldbetouse a buf fered channel.

Thecraw ler is now hig hly con cur rentand printsastorm ofURLs, but it has two pro blems.

The ﬁrs t problem manifests its elf as erro r mess ages in the log afterafew second s of operat ion:

$gobuild gopl.io/ch8/crawl1

$./crawl1 http://gopl.io/

http://gopl.io/

https://golang.org/help/

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

SECTION 8.6. EXAMPLE: CONCURRENT WEB CRAWLER 241

https://golang.org/doc/

https://golang.org/blog/

...

2015/07/15 18:22:12 Get ...: dial tcp: lookup blog.golang.org: no such host

2015/07/15 18:22:12 Get ...: dial tcp 23.21.222.120:443: socket:

too many open files

...

Theinitial erro r mess age isasur prisingrep ort ofaDNS lookup fai lurefor a reliabledom ain.

Thesubsequenterror message reveals the cause:the program create d so manynet wor k con-

ne ction s at oncethatitexceeded the per-pro cesslimitonthe numberofopenﬁles, causing

op erat ions such as DNS lookupsand cal lsto net.Dial to start fai ling.

Theprogram is to o paral lel. Unb ounde d paral lelism israrelyagood ide a since there isalways

alimiting fac tor inthe system, suchasthe numberofCPU coresfor compute-b ound

work loads,the numberofspind les andheads for local disk I/O operat ions,the bandw idt h of

thenet wor k forstreamingdow nlo ads, orthe ser vingcap acity ofa web ser vice.The solut ion is

to limitthe numberofparal lelusesofthe res ource tomatch the level ofparal lelism thatis

avai lable.Asimpleway todothatinour exampleistoens ure thatnomorethan n callsto

links.Extract areactiveatonce, where n is comfortably lessthanthe ﬁle descr ipt or

limit—20, say.Thisisanalogou s to the way a door man at a crow ded nig htclub admits a guest

on lywhensom e ot her guestleaves.

We can limitparal lelism usingabuf fered channel ofcap acity n to model a con cur rency primi-

tive cal le d a counting semap hore.Con ceptu ally, eachofthe n vac antslots in the channel buf fer

repres ents a token entitlingthe holder topro ceed.Sendingavalue into the channel acquires a

token, andreceiving a value fro m thechannel releasesatoken, creating a new vac antslot.

Thisens uresthatatmost n send s canocc ur withoutanint erveningreceive . (A lthough it

mig htbemoreint uit ive totre at ﬁll ed slotsinthe channel buf fer as tokens, usingvac antslots

avoids the need toﬁll the channel buf fer aftercre ating it.) Since the channel elementtyp e is

notimp ortant, we’ll use struct{},whichhas size zero.

Let’srewrite the crawl func tionsothatthe cal l to links.Extract is brackete d by operat ions

to acquireand release a token, thu s ensuring thatatmost20cal lstoitare activeatone time.

It’sgood prac tice tokeepthe semaphore operat ions asclos e as possibletothe I/O operat ion

they regu late.

gopl.io/ch8/crawl2

// tokens is a counting semaphore used to

// enforce a limit of 20 concurrent requests.

var tokens = make(chan struct{}, 20)

func crawl(url string) []string {

fmt.Println(url)

tokens <- struct{}{} // acquire a token

list, err := links.Extract(url)

<-tokens // release the token

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

242 CHAPTER 8. GOROUTINES AND CHANNELS

if err != nil {

log.Print(err)

}

return list

}

Thesecon d problem isthatthe program never ter minates, evenwhenithas discov ere d al l the

lin ks re ach ablefro m theinitial URLs. (Of course,you’re unlikelytonot ice thispro blem unless

youcho ose the initial URLs caref ullyorimp lementthe depth-limiting feature ofExercis e 8.6.)

Fo r theprogram toter minate, weneed tobre akout of the main loop whenthe wor klist is

empt y an d no craw l goro utinesare active.

func main() {

worklist := make(chan []string)

var n int // number of pending sends to worklist

// Start with the command-line arguments.

n++

go func() { worklist <- os.Args[1:] }()

// Crawl the web concurrently.

seen := make(map[string]bool)

for ; n > 0; n-- {

list := <-worklist

for _, link := range list {

if !seen[link] {

seen[link] = true

n++

go func(link string) {

worklist <- crawl(link)

}(link)

}

In thisversion, the count er n ke eps trackofthe numberofsends tothe wor klist thatare yet to

occur. Eachtimeweknowthatanitemneedstobesenttothe wor klist,weincrement n,once

before wesendthe initial command-linearguments, andagain eachtimewestart a craw ler

goro utine.The main loop ter minates when n fal lstozero, since there isnomorewor k to be

done.

No w thecon cur rentcraw ler runs about 20times fasterthanthe bre adt h-ﬁrs t craw ler from

Section5.6, wit houterror s,and ter minates correc tly ifitshouldcompleteits task.

Theprogram below shows an alternat ive solut ion tothe pro blem ofexcessive con cur rency.

Thisversionusesthe original crawl func tionthathas nocount ing semaphore , butcal lsit

from oneof20lon g-live d craw ler goro utines, thu s ensuring thatatmost 20 HTTPrequests are

ac tivecon cur rently.

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

SECTION 8.6. EXAMPLE: CONCURRENT WEB CRAWLER 243

gopl.io/ch8/crawl3

func main() {

worklist := make(chan []string) // lists of URLs, may have duplicates

unseenLinks := make(chan string) // de-duplicated URLs

// Add command-line arguments to worklist.

go func() { worklist <- os.Args[1:] }()

// Create 20 crawler goroutines to fetch each unseen link.

for i := 0; i < 20; i++ {

go func() {

for link := range unseenLinks {

foundLinks := crawl(link)

go func() { worklist <- foundLinks }()

}

}()

}

// The main goroutine de-duplicates worklist items

// and sends the unseen ones to the crawlers.

seen := make(map[string]bool)

for list := range worklist {

for _, link := range list {

if !seen[link] {

seen[link] = true

unseenLinks <- link

}

Thecraw ler goro utinesare all fed bythe samechannel, unseenLinks.The main goroutine is

resp onsible for de-dup lic atingitems itreceivesfro m thewor klist,and thensendingeach

unseen one overthe unseenLinks ch annel toacraw ler goro utine.

The seen mapis conﬁne d within the main goroutine;thatis, itcan beaccessedonlybythat

goro utine.Likeother for msofinfor mat ionhiding, conﬁnement helps usreasonabout the

correc tnessofaprogram. For example, local variables cannot bemention edbynamefro m

outside the functioninwhichthe y aredeclare d;var iables thatdonot escap e (§2.3.4) fro m a

func tioncannot beaccessedfro m outside thatfunction; andenc apsulated ﬁelds ofanobj e ct

cannot beaccessedexceptbythe methodsofthatobj e ct. Inall cas es, infor mat ionhidinghelps

to limitunintended int erac tions bet weenpar ts of the program.

Linksfound by crawl aresenttothe wor klist fro m adedic ated goroutine toavoid dead lock.

To s avespace,wehavenot addressedthe pro blem ofter minat ion in thisexample.

Exercis e 8.6: Adddepth-limiting tothe con cur rentcraw ler.Thatis, if the usersets -depth=3,

then onlyURLs reach able by at mostthree lin ks wi l l be fetch ed.

Exercis e 8.7: Wr ite a con cur rentprogram thatcre atesalocal mirro r of a web sit e , fetching

each reach ablepageand writing ittoadirec tor y on the local disk.Onlypages wit hin the

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

244 CHAPTER 8. GOROUTINES AND CHANNELS

or iginaldom ain (for ins tance, golang.org)shouldbefetch ed. URLs wit hin mirro redpages

shouldbealt ere d as needed sothatthe y refertothe mir rored page , notthe original.

8.7. Multiplexing with select

Theprogram below doesthe count dow n forarocketlaunch.The time.Tick func tionretur ns

achannel onwhichitsends eventsper iodic ally,actinglikea met ron om e.The value ofeach

eventisatimestamp,but it israrely as int erest ing asthe fac t of itsdeliver y.

gopl.io/ch8/countdown1

func main() {

fmt.Println("Commencing countdown.")

tick := time.Tick(1 * time.Second)

for countdown := 10; countdown > 0; countdown-- {

fmt.Println(countdown)

<-tick

}

launch()

}

No w let’sadd the abi lit y to abort the launch sequence bypressingthe retur n ke y during the

countdow n.First,westart a goroutine thattries toreadasinglebytefro m thestand ard inp ut

and, ifitsucce e ds, sends a value onachannel cal le d abort.

gopl.io/ch8/countdown2

abort := make(chan struct{})

go func() {

os.Stdin.Read(make([]byte, 1)) // read a single byte

abort <- struct{}{}

}()

No w each iterat ionofthe count dow n lo opneedstowaitfor anevent toarrive onone ofthe

twochannel s:the tickerchannel ifeverythingisﬁne (‘‘nominal’’ in NASAjargon) oranabort

eventifthere was an ‘‘anom aly.’’ We can’tjustreceive fro m each channel because whiche ver

op erat ionwetry ﬁrs t wi l l blockunt i l comp let ion. Weneed to mu ltipl ex thes e op erat ions,and

to dothat, weneed a select statement.

select {

case <-ch1:

// ...

case x := <-ch2:

// ...use x...

case ch3 <- y:

// ...

default:

// ...

}

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

SECTION 8.7. MULTIPLEXING WITH SELECT 245

Thegeneral for m of a selec t st atement isshown above . Like a switch statement,ithas a num-

berofcas es andanopt ion al default.Eachcas e sp eciﬁes a commu nicati on (a sendorreceive

op erat iononsom e ch annel)and anass oci ated blo ckofstatements. A re ceive expressionmay

appear onits own,asinthe ﬁrs t case,orwit hin a short var iable decl arat ion, as in the secon d

case;the secon d form lets you refer tothe receive d value.

A select waits until a communic ationfor som e case isready topro ceed.Itthenper for ms

that communic ationand exe cut esthe cas e’sass oci ated statements; the other communic ations

do not happen. A select with nocas es, select{},waits forever.

Let’sretur n to our rocketlaunch program. The time.After func tionimmediate lyretur nsa

ch annel,and startsanew goroutine thatsends a singlevalue onthatchannel after the speci-

ﬁed time.The selec t st atement below waits until the ﬁrs t of two eventsarrives, either an abor t

eventorthe event indic atingthat10secon dshaveelaps ed. If10secon dsgobywit h no abort,

thelaunch pro ceeds.

func main() {

// ...create abort channel...

fmt.Println("Commencing countdown. Press return to abort.")

select {

case <-time.After(10 * time.Second):

// Do nothing.

case <-abort:

fmt.Println("Launch aborted!")

return

}

launch()

}

Theexamplebelow ismoresubtle. The channel ch,whose buf fer size is1,isalt ernatelyemp ty

then full,soonlyone ofthe cas es canpro ceed,eit her the sendwhen i is even, orthe receive

when i is odd.Italways prints 02468.

ch := make(chan int, 1)

for i := 0; i < 10; i++ {

select {

case x := <-ch:

fmt.Println(x) // "0" "2" "4" "6" "8"

case ch <- i:

}

If multiplecas es areready, select picksone atrandom, whichens uresthatevery channel has

an equ alchanceofbeingselec ted.Incre asingthe buf fer size ofthe pre vious examplemakes its

output non deter minist ic, because whenthe buf fer isneither full nor emp ty, the selec t st ate-

ment ﬁgurat ive lytossesacoin.

Let’smakeour launch program print the count dow n.The selec t st atement below causeseach

it erat ionofthe looptowaitupto1secon d for an abort,but nolon g er.

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

246 CHAPTER 8. GOROUTINES AND CHANNELS

gopl.io/ch8/countdown3

func main() {

// ...create abort channel...

fmt.Println("Commencing countdown. Press return to abort.")

tick := time.Tick(1 * time.Second)

for countdown := 10; countdown > 0; countdown-- {

fmt.Println(countdown)

select {

case <-tick:

// Do nothing.

case <-abort:

fmt.Println("Launch aborted!")

return

}

launch()

}

The time.Tick func tionbeh avesasifitcre atesagoroutine thatcal ls time.Sleep in a loop,

sendinganevent eachtimeitwakes up. Whenthe count dow n func tionabove retur ns, itstops

re ceiving eventsfro m tick,but the tickergoroutine isstill there , tr yinginvain tosendona

ch annel fro m whichnogoroutine isreceiving—a goro utine leak (§8.4.4).

The Tick func tionisconvenient, but it’sappro priateonlywhenthe ticks will beneeded

thro ughoutthe lifet imeofthe app lic ation. Other wis e,weshoulduse thispattern:

ticker := time.NewTicker(1 * time.Second)

<-ticker.C // receive from the ticker'schannel

ticker.Stop() // cause the ticker'sgoroutine to terminate

Sometimeswewanttotry tosendorreceive onachannel but avoid blo cking ifthe channel is

notready—a non-blocki n g communic ation. A sele ctstatement can do thattoo.Aselect

mayhaveadefault,whichspeciﬁes whattodowhennon e of the other communic ations can

proceed immediate ly.

Theselec t st atement below receivesavalue fro m the abort ch annel ifthere isone toreceive;

ot her wis e it doesnot hing. Thisisanon-blo cking receive operat ion; doingitrep eatedlyis

called polling achannel.

select {

case <-abort:

fmt.Printf("Launch aborted!\n")

return

default:

// do nothing

}

Thezerovalue for a channel is nil.Perhaps sur prisingly, nil channel s aresom etimesuseful.

Becaus e send and receive operat ions onanil channel blo ckforever,acas e in a selec t st atement

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

SECTION 8.8. EXAMPLE: CONCURRENT DIRECTORYTRAVERSAL 247

whos e ch annel isnil isnever selec ted.Thislets ususe nil to enableordis ablecas es that cor-

resp ond tofeatureslikehandlingtimeoutsorcancellat ion, respondingtoother inputevents,

or emitt ing out put.We’ll see anexample in the next sec tion.

Exercis e 8.8: Usingaselec t st atement,add a timeout tothe ech o server fro m Section8.3 so

that itdis conne cts anyclientthatshoutsnot hingwit hin 10 secon ds.

8.8. Example: ConcurrentDirectory Traversal

In thissec tion,we’ll bui ld aprogram thatrep ortsthe diskusage ofone ormoredirec tor ies

sp eciﬁed onthe command line, likethe Unix du command. Mostofits wor k is don e by the

walkDir func tionbelow,whichenumerates the ent ries ofthe direc tor y dir usingthe dirents

helperfunction.

gopl.io/ch8/du1

// walkDir recursively walks the file tree rooted at dir

// and sends the size of each found file on fileSizes.

func walkDir(dir string, fileSizes chan<- int64) {

for _, entry := range dirents(dir) {

if entry.IsDir() {

subdir := filepath.Join(dir, entry.Name())

walkDir(subdir, fileSizes)

}else {

fileSizes <- entry.Size()

}

// dirents returns the entries of directory dir.

func dirents(dir string) []os.FileInfo {

entries, err := ioutil.ReadDir(dir)

if err != nil {

fmt.Fprintf(os.Stderr, "du1: %v\n", err)

return nil

}

return entries

}

The ioutil.ReadDir func tionretur nsa slice of os.FileInfo—thesameinfor mat ionthata

call to os.Stat returnsfor a single ﬁle.For eachsub direc tor y, walkDir re cursive lycal lsits elf,

andfor each ﬁle, walkDir send s amessage onthe fileSizes ch annel.The mess age isthe size

of the ﬁle in bytes.

Themain function, shown below,usestwo goro utines. Thebackg round goroutine cal ls

walkDir foreachdirec tor y sp eciﬁed onthe command lineand ﬁnallyclos es the fileSizes

ch annel.The main goroutine computesthe sum ofthe ﬁle sizes itreceivesfro m thechannel

and ﬁnallyprintsthe tot al.

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

248 CHAPTER 8. GOROUTINES AND CHANNELS

// The du1 command computes the disk usage of the files in a directory.

package main

import (

"flag"

"fmt"

"io/ioutil"

"os"

"path/filepath"

)

func main() {

// Determine the initial directories.

flag.Parse()

roots := flag.Args()

if len(roots) == 0 {

roots = []string{"."}

}

// Traverse the file tree.

fileSizes := make(chan int64)

go func() {

for _, root := range roots {

walkDir(root, fileSizes)

}

close(fileSizes)

}()

// Print the results.

var nfiles, nbytes int64

for size := range fileSizes {

nfiles++

nbytes += size

}

printDiskUsage(nfiles, nbytes)

}

func printDiskUsage(nfiles, nbytes int64) {

fmt.Printf("%d files %.1f GB\n", nfiles, float64(nbytes)/1e9)

}

Thisprogram pausesfor a lon g whilebeforeprint ing its result:

$gobuild gopl.io/ch8/du1

$./du1 $HOME /usr /bin /etc

213201 files 62.7 GB

Theprogram wou ldbenicer if itkeptusinfor med ofits progress. How ever, simply mov ing

the printDiskUsage call int o theloopwou ldcause ittoprint thous and s of lines ofout put.

Thevar iantof du belowprintsthe tot als per iodic ally,but only ifthe -v ﬂag isspeciﬁed since

notall users will wanttosee progressmessages. Thebackg round goroutine thatloops over

roots remainsunchange d.The main goroutine nowusesatickertogenerateeventsevery

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

SECTION 8.8. EXAMPLE: CONCURRENT DIRECTORYTRAVERSAL 249

500ms, andaselec t st atement towaitfor eit her a ﬁle size message , in whichcas e it updatesthe

totals,oratickevent,inwhichcas e it prints the cur renttot als.Ifthe -v ﬂag isnot speciﬁed,

the tick ch annel remainsnil,and its cas e in the select is effec tive lydis abled.

gopl.io/ch8/du2

var verbose = flag.Bool("v", false, "show verbose progress messages")

func main() {

// ...start background goroutine...

// Print the results periodically.

var tick <-chan time.Time

if *verbose {

tick = time.Tick(500 * time.Millisecond)

}

var nfiles, nbytes int64

loop:

for {

select {

case size, ok := <-fileSizes:

if !ok {

break loop // fileSizes was closed

}

nfiles++

nbytes += size

case <-tick:

printDiskUsage(nfiles, nbytes)

}

printDiskUsage(nfiles, nbytes) // final totals

}

Sincethe program nolon g erusesarange lo op, the ﬁrs t select case mustexp

licitlytest

whet her the fileSizes ch annel has beenclos ed, usingthe two-resultfor m of receive opera-

tion.Ifthe channel has beenclos ed, the program bre aks out of the loop. The lab ele d break

st atement bre aks out of bot h the select andthe for lo op; an unlabele d break wouldbre ak

outofonlythe select,causingthe looptobeg in thenext iterat ion.

Theprogram now gives usaleisure lystreamofupdates:

$gobuild gopl.io/ch8/du2

$./du2 -v $HOME /usr /bin /etc

28608 files 8.3 GB

54147 files 10.3 GB

93591 files 15.1 GB

127169 files 52.9 GB

175931 files 62.2 GB

213201 files 62.7 GB

Ho wever,itstill takes too lon g to ﬁnish.There’snoreasonwhy all the cal lsto walkDir can’tbe

done con cur rently, there byexploit ing paral lelism in the disksystem. Thethirdversionof du,

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

250 CHAPTER 8. GOROUTINES AND CHANNELS

below, cre atesanew goroutine for eachcal l to walkDir.Itusesasync.WaitGroup (§8.5) to

countthe numberofcal lsto walkDir that arestill active, and a clos er goro utine toclos e the

fileSizes ch annel whenthe count erdro pstozero.

gopl.io/ch8/du3

func main() {

// ...determine roots...

// Traverse each root of the file tree in parallel.

fileSizes := make(chan int64)

var n sync.WaitGroup

for _, root := range roots {

n.Add(1)

go walkDir(root, &n, fileSizes)

}

go func() {

n.Wait()

close(fileSizes)

}()

// ...select loop...

}

func walkDir(dir string, n *sync.WaitGroup, fileSizes chan<- int64) {

defer n.Done()

for _, entry := range dirents(dir) {

if entry.IsDir() {

n.Add(1)

subdir := filepath.Join(dir, entry.Name())

go walkDir(subdir, n, fileSizes)

}else {

fileSizes <- entry.Size()

}

Sincethisprogram createsmanythous and s of goroutinesatits peak, wehavetochange

dirents to use a count ing semaphore topre ventitfro m op eningtoo many ﬁles at once, just

as wedid for the web craw ler in Sec tion 8.6:

// sema is a counting semaphore for limiting concurrency in dirents.

var sema = make(chan struct{}, 20)

// dirents returns the entries of directory dir.

func dirents(dir string) []os.FileInfo {

sema <- struct{}{} // acquire token

defer func() { <-sema }() // release token

// ...

Thisversionrunsseveral times fasterthanthe pre vious one,thoug h thereisalot ofvar iabilit y

from systemtosystem.

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

SECTION 8.9. CANCELLATION 251

Exercis e 8.9: Wr ite a versionof du that comp utesand per iodic ally displ ays sep aratetot als for

each ofthe root direc tor ies.

8.9. Cancellation

Sometimesweneed toins truct a goroutine tostopwhatitisdoing , forexample, inaweb

server per for mingacomputation on beh alf ofaclientthathas disconne cte d.

Thereisnoway for onegoroutine toter minateanother direc tly,since thatwou ldleave all its

sh are d var iables in undeﬁne d st ates. Inthe rocketlaunch program (§8.7) wesentasingle

value onachannel named abort,whichthe count dow n goro utine int erprete d as a requestto

stop its elf.But what if we need tocanceltwo goro utines, or an arbit rar y numb er?

Onepossibi lit y mig htbetosendasmanyeventsonthe abort ch annel asthere are goroutines

to cancel. Ifsom e of the goroutineshavealready ter minated thems elves, how ever, our count

wi l l be too large , andour sends will get stuck.Onthe other hand, ifthose goroutineshave

sp awned other goroutines, our countwill betoo small,and som e goro utineswill remain

unawareofthe cancellat ion. Ingeneral,it’shardtoknowhow manygoroutinesare wor king

on our beh alf at anygiven mom ent.Moreover, whenagoroutine receivesavalue fro m the

abort ch annel,itcon sumes thatvalue sothatother goroutineswon’tsee it. For cancellat ion,

what weneed isare liablemechanism to broadcast an event overachannel sothatmany

goro utinescan see it as it occ ursand can later see thatit has occurred.

Re call thatafter a channel has beenclos edand drained ofall sentvalues, subsequentreceive

op erat ions pro ceed immediate ly, yieldingzerovalues. Wecan exploit thistocre ate a bro ad-

cast mechanism: don’tsendavalue onthe channel, cl ose it.

We c an add cancellat iontothe du prog ram from the pre vious sec tion wit h afew simple

ch anges. First,wecre ate a cancellat ionchannel onwhichnovalues areeversent, but whose

closureindic ates thatitistimefor the program tostopwhatitisdoing . We als o deﬁne a

ut i lit y func tion, cancelled,thatche cks or poll s thecancellat ionstate atthe ins tantitiscal le d.

gopl.io/ch8/du4

var done = make(chan struct{})

func cancelled() bool {

select {

case <-done:

return true

default:

return false

}

Next, wecre ate a goroutine thatwill readfro m thestand ard inp ut, whichistypic ally con-

ne cte d to the ter minal.Assoonasany inp utisread(forins tance,the userpress esthe retur n

ke y), thisgoroutine bro adc asts the cancellat ionbyclosingthe done ch annel.

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

252 CHAPTER 8. GOROUTINES AND CHANNELS

// Cancel traversal when input is detected.

go func() {

os.Stdin.Read(make([]byte, 1)) // read a single byte

close(done)

}()

No w we need tomakeour goroutinesrespond tothe cancellat ion. Inthe main goroutine,we

add a thirdcas e to the selec t st atement thattries toreceive fro m the done ch annel.The func-

tion retur nsifthiscas e is everselec ted,but beforeitretur nsitmustﬁrs t drain the fileSizes

ch annel,dis cardingall values until the channel isclos ed. Itdoesthistoens ure thatany active

callsto walkDir canrun tocomplet ionwit houtgetting stuck sendingto fileSizes.

for {

select {

case <-done:

// Drain fileSizes to allow existing goroutines to finish.

for range fileSizes {

// Do nothing.

}

return

case size, ok := <-fileSizes:

// ...

}

The walkDir goro utine pol lsthe cancellat ionstatu s when itbeg ins, andretur nswit houtdoing

anyt hing if the statu s is set. Thistur nsall goroutinescre ate d af ter cancellat ionint o no-ops:

func walkDir(dir string, n *sync.WaitGroup, fileSizes chan<- int64) {

defer n.Done()

if cancelled() {

return

}

for _, entry := range dirents(dir) {

// ...

}

It mig htbeproﬁtabletopol l thecancellat ionstatu s again wit hin walkDir’sloop, toavoid cre-

at inggoroutinesafter the cancellat ionevent.Cancellat ioninv olves a trade-off;aquicker

resp ons e of ten requires moreint rusivechangestoprogram logic. Ensuring thatnoexp ensive

op erat ions everocc ur af ter the cancellat ionevent may requireupdat ingmanyplaces in your

co de,but oftenmostofthe beneﬁtcan beobt ained byche cking for cancellat ioninafew

importantplaces.

Alit tle proﬁlingofthisprogram reveale d that the bot tleneckwas the acquisition of a sema-

ph ore token in dirents.The select belowmakes thisoperat ioncancellable andreduces the

typicalcancellat ionlatency ofthe program fro m hundre dsofmillis econd s to tens:

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

SECTION 8.10. EXAMPLE: CHAT SERVER 253

func dirents(dir string) []os.FileInfo {

select {

case sema <- struct{}{}: // acquire token

case <-done:

return nil // cancelled

}

defer func() { <-sema }() // release token

// ...read directory...

}

No w,whencancellat ionocc urs, allthe backg round goroutinesquicklystopand the main func-

tion retur ns. Of course,when main returns, a program exits, soitcan behardtotel l amain

func tionthatcle ansupafter its elf fro m on e that does not.There’sahandy trick wecan use

during testing:ifins teadofretur ningfro m main in the event ofcancellat ion, weexe cut e acal l

to panic,thenthe runtime will dumpthe stack ofevery goroutine inthe program. Ifthe main

goro utine isthe onlyone lef t, then ithas cle aned upafter its elf.But ifother goroutines

remain, the y maynot havebeenpro perly cancelled,orperhaps the y have beencancelled but

thecancellat iontakes time; a litt leinv est igat ionmay bewor thw hile. The panic dumpoften

cont ainssufﬁcientinfor mat iontodistinguish these cas es.

Exercis e 8.10: HT TP re quests may becancelled byclosingthe opt ion al Cancel ch annel inthe

http.Request st ruc t. Mo dif y theweb craw ler of Sec tion 8.6 tosup por t cancel lat ion.

Hint:the http.Get conv enience functiondoesnot giveyou anopp ortunity tocustomize a

Request.Ins tead, cre ate the requestusing http.NewRequest,set its Cancel ﬁeld, thenper-

form therequestbycal ling http.DefaultClient.Do(req).

Exercis e 8.11: Fo llowing the appro ach of mirroredQuery in Sec tion 8.4.4, implementavar i-

antof fetch that requests several URLs conc urrently. Assoonasthe ﬁrs t resp ons e ar r ives,

cancel the other requests.

8.10. Example: Chat Server

We’ll ﬁnish thischapt erwit h achatser ver thatlets several users bro adc asttextualmessagesto

each other.There are fourkinds ofgoroutine inthisprogram. Thereisone ins tance apie ce of

the main and broadcaster goro utines, andfor eachclientconne ction there isone handle-

Conn andone clientWriter goro utine.The bro adc asterisagood illustrat ionofhow select

is used,since ithas torespond tothree dif ferentkinds ofmessages.

Thejob ofthe main goroutine,shown below,istolistenfor and acceptincomingnet wor k con-

ne ction s from clients. For eachone,itcre atesanew handleConn goro utine,justasinthe con-

currentech o server wesaw at the start ofthischapt er.

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

254 CHAPTER 8. GOROUTINES AND CHANNELS

gopl.io/ch8/chat

func main() {

listener, err := net.Listen("tcp", "localhost:8000")

if err != nil {

log.Fatal(err)

}

go broadcaster()

for {

conn, err := listener.Accept()

if err != nil {

log.Print(err)

continue

}

go handleConn(conn)

}

Next isthe bro adc aster. Its localvar iable clients re cords the cur rentset ofconne cte d clients.

Theonlyinfor mat ionrecorde d ab out eachclientisthe identity ofits out going message chan-

nel,about whichmorelater.

type client chan<- string // an outgoing message channel

var (

entering = make(chan client)

leaving = make(chan client)

messages = make(chan string) // all incoming client messages

)

func broadcaster() {

clients := make(map[client]bool) // all connected clients

for {

select {

case msg := <-messages:

// Broadcast incoming message to all

// clients' outgoing message channels.

for cli := range clients {

cli <- msg

}

case cli := <-entering:

clients[cli] = true

case cli := <-leaving:

delete(clients, cli)

close(cli)

}

Thebro adc asterlistens onthe global entering and leaving ch annel s forannouncements of

ar r iving and dep arting clients. Whenitreceivesone ofthese events, itupdates the clients

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

SECTION 8.10. EXAMPLE: CHAT SERVER 255

set, andifthe event was a depar ture, itclos es theclient’sout going message channel.The

broadc asterals o listensfor eventsonthe global messages ch annel,towhicheachclientsends

al l itsincomingmessages. Whenthe bro adc asterreceivesone ofthese events, itbro adc asts the

mess age toevery conne cte d client.

No w let’slookatthe per-clientgoroutines. The handleConn func tioncre atesanew out going

mess age channel for itsclientand announces the arrival ofthisclienttothe bro adc asterover

the entering ch annel.Thenitreads every lineoftext fro m theclient, sendingeachlinetothe

broadc asteroverthe globalincomingmessage channel,preﬁxingeachmessage wit h theiden-

tity ofits sender.Oncethere isnot hingmoretoreadfro m theclient, handleConn announces

thedep arture ofthe clientoverthe leaving ch annel and clos es theconne ction.

func handleConn(conn net.Conn) {

ch := make(chan string) // outgoing client messages

go clientWriter(conn, ch)

who := conn.RemoteAddr().String()

ch <- "You are " + who

messages <- who + " has arrived"

entering <- ch

input := bufio.NewScanner(conn)

for input.Scan() {

messages <- who + ": " + input.Text()

}

// NOTE: ignoring potential errors from input.Err()

leaving <- ch

messages <- who + " has left"

conn.Close()

}

func clientWriter(conn net.Conn, ch <-chan string) {

for msg := range ch {

fmt.Fprintln(conn, msg) // NOTE: ignoring network errors

}

In addition, handleConn createsaclientWriter goro utine for eachclientthatreceivesmes-

sagesbro adc asttothe client’sout going message channel and writesthemtothe client’snet-

work conne ction.The clientwriter’sloopter minates whenthe bro adc asterclos es thechannel

af ter receiving a leaving notiﬁcation.

Thedispl aybelow shows the ser ver in action wit h twoclients in sep aratewindows onthe

same computer, using netcat to chat:

$gobuild gopl.io/ch8/chat

$gobuild gopl.io/ch8/netcat3

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

256 CHAPTER 8. GOROUTINES AND CHANNELS

$./chat &

$./netcat3

You are 127.0.0.1:64208 $./netcat3

127.0.0.1:64211 has arrived You are 127.0.0.1:64211

Hi!

127.0.0.1:64208: Hi! 127.0.0.1:64208: Hi!

Hi yourself.

127.0.0.1:64211: Hi yourself. 127.0.0.1:64211: Hi yourself.

127.0.0.1:64208 has left

$./netcat3

You are 127.0.0.1:64216 127.0.0.1:64216 has arrived

Welcome.

127.0.0.1:64211: Welcome. 127.0.0.1:64211: Welcome.

127.0.0.1:64211 has left

Whilehosting a chatsessionfor n clients, thisprogram runs2n+2 conc urrentlycommunic at-

inggoroutines, yet itneedsnoexplicitlocking operat ions (§9.2). The clients mapiscon-

ﬁne d to a singlegoroutine,the bro adc aster, soitcannot beaccessedcon cur rently. The only

var iables thatare share d by multiplegoroutinesare channel s andins tances of net.Conn,bot h

of whichare concur rency saf e.We’ll tal k more about conﬁnement,con cur rency safet y,and the

implic ations ofsharing var iables acrossgoroutines in the next chapt er.

Exercis e 8.12: Make the bro adc asterannouncethe cur rentset ofclients toeachnew arrival.

Thisrequires thatthe clients setand the entering and leaving ch annel s re cordthe client

name too.

Exercis e 8.13: Make the chatser ver disconne ctidleclients, suchasthose thathavesentno

mess agesinthe last ﬁve minut es. Hint: cal ling conn.Close() in anot her goroutine unb locks

ac tive Read callssuch as the one don e by input.Scan().

Exercis e 8.14: Change the chatser ver’snet wor k prot ocolsothateachclientprovides its name

on ent ering . Us e that nameins teadofthe networ k addresswhenpreﬁxingeachmessage wit h

itssender’sidentity.

Exercis e 8.15: Failureofany clientprogram toreaddat a in a timelymanner ult imate lycauses

al l clients toget stuck.Modif y thebro adc astertoskipamessage rat her thanwaitifaclient

wr iterisnot ready toacceptit. Alt ernat ive ly, add buf fer ingtoeachclient’sout going message

ch annel sothatmostmessagesare not dro p ped;the bro adc astershoulduse a non-blo cking

send tothischannel.

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

Concurrency with

Shared Variables

In the pre vious chapt er, wepresent edseveral programsthatuse goroutinesand channel s to

express con cur rency inadirec t andnatural way.How ever, indoing so, weglossedovera

numb erofimp ortantand subtleissues thatprogrammersmustbearinmindwhenwriting

conc urrentcode.

In thischapt er, we’ll takeaclos er lo okatthe mech anics ofcon cur rency.Inpar tic ular, we’ll

pointout som e of the pro blemsass oci ated wit h sh aring var iables amon g mu ltiplegoroutines,

theanalyticaltechniques for recog nizingthose pro blems, andthe patternsfor solv ing them.

Final ly, we’ll explain som e of the technic al dif ferences bet weengoroutinesand operat ingsys-

temthreads.

9.1. RaceConditions

In a sequential program, thatis, a program wit h on lyone goroutine,the steps ofthe program

happ eninthe fami liar exec ution order deter mined bythe program logic. For ins tance,ina

sequence ofstatements, the ﬁrs t on e happ ens beforethe secon d on e,and soon. Inaprogram

with two ormoregoroutines, the steps wit hin eachgoroutine happeninthe fami liar order,but

in general wedon’tknowwhether an event x in one goroutine happens beforeanevent y in

anot her goroutine,orhappens after it, orissimultane ous wit h it.Whenwecannot conﬁdently

saythatone event happ ens before theother,thenthe events x and y are concur rent.

Consider a functionthatwor kscor rec tly inasequential program. That functionis concur-

renc y-s afe if itcontinues towor k correc tly evenwhencal le d conc urrently, thatis, fro m twoor

more goroutineswit h no addition alsynchro nizat ion. Wecan generalize thisnot ion toaset of

257

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

258 CHAPTER 9. CONCURRENCY WITH SHARED VARIABLES

collab orat ingfunctions,suchasthe methodsand operat ions ofapar tic ulartyp e.Atype is

conc urrency-s afe if all its accessiblemet hodsand operat ions are con cur rency-s afe.

We can makeaprogram conc urrency-s afewit houtmak ingevery con crete typ e in that

prog ram conc urrency-s afe. Indeed,con cur rency-s afetyp es arethe exception rat her thanthe

rule,soyou shouldaccessavar iable conc urrentlyonlyifthe documentation for itstyp e says

that thisissafe. Weavoid conc urrentaccesstomostvar iables either by conﬁni n g them toa

singlegoroutine orbymaintainingahig her-le vel invar iantof mu tualexclu sion.We’ll explain

thes e terms in thischapt er.

In contrast, expor ted package-le vel functions are general lyexp ected tobecon cur rency-s afe.

Sincepackage-le vel var iables cannot beconﬁne d to a singlegoroutine,functions thatmodif y

them mustenforce mut ual exclusion.

Thereare manyreasons a functionmig htnot wor k when cal le d conc urrently, includingdead-

lo ck, live lock, andres ource starvat ion. Wedon’thavespace todis cussall ofthem, sowe’ll

fo cus onthe most imp ortantone,the ra c e condit ion.

Arace condit ion isasit uat ioninwhichthe program does not givethe cor rec t resu ltfor som e

interleavingsofthe operat ions ofmultiplegoroutines. Race condit ion s areper nicious because

they may remain latentinaprogram andapp ear infre quently, perhaps onlyunder heavy load

or whenusingcer tain comp ilers, platfor ms, orarc hit ectures. Thismakes themhardto

reproduce anddiagnos e.

It istradition altoexplain the ser iousness ofrace condit ion s thro ugh the metaph orofﬁnancial

loss, sowe’ll con sider a simpleban k accountprogram.

// Package bank implements a bank with only one account.

package bank

var balance int

func Deposit(amount int) { balance = balance + amount }

func Balance() int { return balance }

(Wecou ldhavewritt enthe bodyofthe Deposit func tionas balance += amount,whichis

equivalent, but the lon g erfor m wi l l simplif y theexplanation.)

Fo r aprogram thistrivial, wecan see ataglancethatany sequence ofcal lsto Deposit and

Balance wi l l give the rig htanswer, thatis, Balance wi l l report the sum ofall amounts

previously dep osite d. Ho wever,ifwecal l thes e func tions not inasequence but con cur rently,

Balance is nolon g erguarante e d to givethe rig htanswer. Con sider the fol low ing two

goro utines, whichrepresent two transactions onajoint ban k account:

// Alice:

go func() {

bank.Deposit(200) // A1

fmt.Println("=", bank.Balance()) // A2

}()

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

SECTION 9.1. RACE CONDITIONS 259

// Bob:

go bank.Deposit(100) // B

Alice deposits $200, thenche cks her bal ance,whi le Bobdep osits $100. Sincethe steps A1 and

A2 occurcon cur rentlywit h B,wecannot pre dic t theorder in whichthe y happ en. Int uit ive ly, it

mig htseemthatthere are onlythree possibleorder ings, whichwe’ll cal l ‘‘Alice ﬁrs t,’’ ‘‘Bob

ﬁrs t,’’ and ‘‘Alice/B ob/A lice.’’ Thefol low ing table shows the value ofthe balance var iable after

each step. The quote d st rings represent the print edbal ance slips.

Alice first Bob first Alice/Bob/Alice

000

A1 200 B100 A1 200

A2 "= 200" A1 300 B300

B300 A2 "= 300" A2 "= 300"

In all cas es theﬁnalbal ance is$300. Theonlyvar iat ioniswhether Alice’sbal ance slipincludes

Bob’stransactionornot,but the customers are sat isﬁed eit her way.

Butthisint uit ion iswro ng. There isafourthpossibleout com e,inwhichBob’sdep ositocc urs

in the midd le of Alice’sdep osit, afterthe bal ance has beenread(balance + amount)but before

it has beenupdated (balance = ...), causingBob’stransactiontodis app ear.Thisisbecause

Alice’sdep ositoperat ion A1 is reallyasequence oftwo operat ions,areadand a write;cal l them

A1r and A1w.Here’sthe pro blemat ic interleaving:

Data race

A1r 0 ... = balance + amount

B100

A1w 200 balance = ...

A2 "= 200"

Af ter A1r,the expression balance + amount evaluatesto200, sothisisthe value writt endur-

ing A1w,despit e theint erveningdep osit. The ﬁnalbal ance isonly$200. Theban k is $100

rich er at Bob’sexp ense.

Thisprogram cont ainsapar tic ularkindofrace condit ion cal le d a data race.Adata race

occurs whene ver two goro utinesaccessthe samevar iable conc urrentlyand atleast one ofthe

accessesisawrite.

Things get evenmessier if the dat a race invo l vesavar iable ofatyp e that islargerthanasingle

machineword, suchasanint erface,astr ing , or a slice.Thiscodeupdates x conc urrentlyto

twoslices ofdif ferentlengt hs:

var x []int

go func() { x = make([]int, 10) }()

go func() { x = make([]int, 1000000) }()

x[999999] = 1 // NOTE: undefined behavior; memory corruption possible!

Thevalue of x in the ﬁnalstatement isnot deﬁne d;itcou ldbenil,oraslice oflengt h 10, ora

slice oflengt h 1,000,000. But recal l that there are three partstoaslice: the point er, the lengt h,

andthe cap acity.Ifthe point ercom esfro m theﬁrs t call to make andthe lengt h comesfro m

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

260 CHAPTER 9. CONCURRENCY WITH SHARED VARIABLES

thesecon d, x wouldbea chimera, a slice whose nominal lengt h is 1,000,000 but whose under-

ly ing array has only10elements. Inthisevent uality,storing toelement999,999 wou ldclobb er

an arbit rar y faraway memor y lo cat ion, wit h cons equences thatare imp ossible topre dic t and

hard todebug andlocalize.Thissemanticmineﬁeldiscal le d un deﬁne d behavior andiswel l

know n to C programmers; for tunatelyitisrarely as tro ubles ome in Go as in C.

Ev enthe notion thatacon cur rentprogram isanint erleaving ofseveral sequential programsis

afalse int uit ion.Aswe’ll see inSec tion 9.4, dat a races may haveevenstrangerout com es.

Many programmers—even som e very cle ver ones—wi l l occasionallyoffer justiﬁcation s for

know n data races in their programs: ‘‘thecostofmut ual exclusionistoo hig h,’’ ‘‘this log ic is

on lyfor log ging,’’ ‘‘Idon’tmindifI dro p some mess ages,’’ andsoon. Theabs ence ofpro blems

on a given comp iler andplatfor m maygivethemfalse conﬁdence.Ago o d rule ofthumb is

that th ere isnosuchthi n g as a benig n data race.Sohow doweavoid dat a races in our

prog rams?

We’ll rep eat the deﬁnition,since itissoimp ortant: A dat a race occ urswhene ver two

goro utinesaccessthe samevar iable conc urrentlyand atleast one ofthe accessesisawrite. It

fo llows fro m this deﬁnition thatthere are three ways toavoid a dat a race.

The ﬁrs t way isnot towrite the var iable.Con sider the map below,whichislazi lypopu lated as

each key isrequeste d forthe ﬁrs t time.If Icon is cal le d sequential ly, the program wor ksﬁne,

butif Icon is cal le d conc urrently, there isadat a race accessingthe map.

var icons = make(map[string]image.Image)

func loadIcon(name string) image.Image

// NOTE: not concurrency-safe!

func Icon(name string) image.Image {

icon, ok := icons[name]

if !ok {

icon = loadIcon(name)

icons[name] = icon

}

return icon

}

If ins teadweinitialize the map wit h al l ne cessary ent ries beforecre ating addition algoroutines

andnever modif y it again, thenany numberofgoroutinesmay safelycal l Icon conc urrently

since eachonlyreads the map.

var icons = map[string]image.Image{

"spades.png": loadIcon("spades.png"),

"hearts.png": loadIcon("hearts.png"),

"diamonds.png": loadIcon("diamonds.png"),

"clubs.png": loadIcon("clubs.png"),

}

// Concurrency-safe.

func Icon(name string) image.Image { return icons[name] }

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

SECTION 9.1. RACE CONDITIONS 261

In the exampleabove , the icons var iable isassig ned dur ingpackageinitializat ion, which

happ ens before theprogram’s main func tionstartsrunning. Onceinitialize d, icons is never

mo diﬁed.Dat a st ruc tures thatare never modiﬁed orare immut ableare inherentlycon cur-

renc y-s afeand need nosynchro nizat ion. But obv iou sly wecan’tuse thisappro ach ifupdates

areess ent ial, as wit h aban k account.

Thesecon d way toavoid a dat a race istoavoid accessingthe var iable fro m mu ltiple

goro utines. Thisisthe appro ach taken bymanyofthe programsinthe pre vious chapt er. For

example, the main goroutine inthe con cur rentweb craw ler (§8.6) isthe solegoroutine that

accessesthe seen map, and the broadcaster goro utine inthe chatser ver (§8.10) isthe only

goro utine thataccessesthe clients map. These var iables are conﬁne d to a singlegoroutine.

Sinceother goroutinescannot accessthe var iable direc tly,the y mu stuse a channel tosendthe

conﬁninggoroutine a requesttoquer y or update the var iable.Thisiswhatismeant bythe Go

mant ra ‘‘Do not communic atebysharing memor y;ins tead, share memor y by communic at-

ing.’’ Agoroutine thatbro kersaccesstoaconﬁne d var iable usingchannel requests iscal le d a

moni tor goroutine forthatvar iable.For example, the broadcaster goro utine monitors access

to the clients map.

Here’sthe ban k examplerewritt enwit h the balance var iable conﬁne d to a monit or goro utine

called teller:

gopl.io/ch9/bank1

// Package bank provides a concurrency-safe bank with one account.

package bank

var deposits = make(chan int) // send amount to deposit

var balances = make(chan int) // receive balance

func Deposit(amount int) { deposits <- amount }

func Balance() int {return <-balances }

func teller() {

var balance int // balance is confined to teller goroutine

for {

select {

case amount := <-deposits:

balance += amount

case balances <- balance:

}

func init() {

go teller() // start the monitor goroutine

}

Ev enwhenavar iable cannot beconﬁne d to a singlegoroutine for itsent ire lifet ime, conﬁne-

ment may still beasolut ion tothe pro blem ofcon cur rentaccess. For example, it’scommonto

sh are a var iable bet weengoroutinesinapip elinebypassingits addressfro m on e st age tothe

next overachannel.Ifeachstage ofthe pip elineref rainsfro m accessingthe var iable after

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

262 CHAPTER 9. CONCURRENCY WITH SHARED VARIABLES

sendingittothe next stage , then allaccessestothe var iable aresequential.Ineffec t,the var i-

able isconﬁne d to one stage ofthe pip eline, thenconﬁne d to the next, andsoon. Thisdis ci-

plineissom etimescal le d serial conﬁnement.

In the examplebelow, Cakesare ser ial lyconﬁne d,ﬁrs t to the baker goro utine,thentothe

icer goro utine:

type Cake struct{ state string }

func baker(cooked chan<- *Cake) {

for {

cake := new(Cake)

cake.state = "cooked"

cooked <- cake // baker never touches this cake again

}

func icer(iced chan<- *Cake, cooked <-chan *Cake) {

for cake := range cooked {

cake.state = "iced"

iced <- cake // icer never touches this cake again

}

Thethirdway toavoid a dat a race istoallow manygoroutinestoaccessthe var iable,but only

on e at a time. Thisappro ach isknown as mu tualexclu sion andisthe subjec t of the next

section.

Exercis e 9.1: Addafunction Withdraw(amount int) bool to the gopl.io/ch9/bank1

prog ram. Theresultshouldindic atewhether the transactionsucce e de d or fai le d duetoins uf-

ﬁcientfunds.The mess age senttothe monitorgoroutine mustcontain bot h theamountto

withdraw andanew channel overwhichthe monitorgoroutine can sendthe boole an resu lt

back to Withdraw.

9.2. Mutual Exclusion: sync.Mutex

In Sec tion 8.6, weusedabuf fered channel asacounting semap hore to ens ure thatnomore

than 20 goroutinesmade simultane ous HTTPrequests. Wit h thesameide a, we can use a

ch annel ofcap acity 1 toens ure thatatmostone goroutine accessesashare d var iable at a time.

Asemaphore thatcountsonlyto1iscal le d a binary semap hore.

gopl.io/ch9/bank2

var (

sema =make(chan struct{}, 1) // a binary semaphore guarding balance

balance int

)

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

SECTION 9.2. MUTUAL EXCLUSION: SYNC.MUTEX 263

func Deposit(amount int) {

sema <- struct{}{} // acquire token

balance = balance + amount

<-sema // release token

}

func Balance() int {

sema <- struct{}{} // acquire token

b:=balance

<-sema // release token

return b

}

Thispattern of mu tualexclu sion is sousefulthatitissup por ted direc tly bythe Mutex type

from the sync package. Its Lock method acquires the token (cal le d a lock)and its Unlock

method releasesit:

gopl.io/ch9/bank3

import "sync"

var (

mu sync.Mutex // guards balance

balance int

)

func Deposit(amount int) {

mu.Lock()

balance = balance + amount

mu.Unlock()

}

func Balance() int {

mu.Lock()

b:=balance

mu.Unlock()

return b

}

Each timeagoroutine accessesthe var iables ofthe ban k (just balance here), itmustcal l the

mu tex’s Lock method toacquireanexc lusivelock. Ifsom e ot her goroutine has acquired the

lo ck, thisoperat ionwill blo ckunt i l theother goroutine cal ls Unlock andthe lockbecom es

avai lable again. Themut ex guards theshare d var iables. Byconvent ion,the var iables guarde d

by a mut exare declare d immediate lyafter the declarat ionofthe mut exits elf.Ifyou deviate

from this, besuretodocumentit.

Thereg ionofcodebet ween Lock and Unlock in whichagoroutine isfre e to readand modif y

theshare d var iables iscal le d a cr iti cal sec tion.The lockholder’scal l to Unlock happ ens before

anyother goroutine can acquirethe lockfor itself.Itisess ent ial thatthe goroutine release the

lo ckonceitis ﬁnishe d,onall pathsthrough the function, includingerror pat hs.

Theban k prog ram ab ove exemp liﬁes a common con cur rency pattern.Asetofexp orted func-

tion s encapsulates one ormorevar iables sothatthe onlyway toaccessthe var iables isthrough

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

264 CHAPTER 9. CONCURRENCY WITH SHARED VARIABLES

thes e func tions (or met hods, for the var iables ofanobj e ct). Eachfunctionacquires a mut ex

lo ckatthe beg inningand releasesitatthe end,there byens uring thatthe shared var iables are

notaccessedcon cur rently. Thisarrangement offunctions,mut exlock, andvar iables iscal le d

a moni tor.(Thisolder use ofthe word ‘‘monitor’’ inspired the ter m ‘‘monitorgoroutine.’’ Both

us esshare the meaningofabro ker thatens uresvar iables areaccessedsequential ly.)

Sincethe criticalsec tion s in the Deposit and Balance func tions are soshort—asingleline, no

branching—calling Unlock at the end isstraig htfor ward. Inmorecomplex crit icalsec tion s,

especi ally those inwhicherror s mu stbedealt wit h by retur ningearly,itcan behardtotel l that

callsto Lock and Unlock arestr ictly paired onall paths. Go’s defer st atement com estothe

rescue: bydefer r ingacal l to Unlock,the criticalsec tion imp licitlyextends tothe end ofthe

currentfunction, fre eingusfro m having toremembertoins ert Unlock callsinone ormore

pl aces far fro m thecal l to Lock.

func Balance() int {

mu.Lock()

defer mu.Unlock()

return balance

}

In the exampleabove , the Unlock exec utes af ter theretur n st atement has readthe value of

balance,sothe Balance func tioniscon cur rency-s afe. Asa bonus,wenolon g erneed the

lo cal variable b.

Furthermore, a defer red Unlock wi l l runevenifthe criticalsec tion panics, whichmay be

important in programsthatmakeuse of recover (§5.10). A defer is marg inallymoreexp en-

sive thananexplicitcal l to Unlock,but not enoug h to justify lesscle ar co de.Asalways wit h

conc urrentprograms, favor clarity and resist premature opt imizat ion. Where possible, use

defer andlet crit icalsec tion s extendtothe end ofafunction.

Consider the Withdraw func tionbelow.Onsuccess, itreduces the bal ance bythe speciﬁed

amount and retur ns true.But ifthe accountholds ins ufﬁcientfunds for the transaction,

Withdraw restores the bal ance andretur ns false.

// NOTE: not atomic!

func Withdraw(amount int) bool {

Deposit(-amount)

if Balance() < 0 {

Deposit(amount)

return false // insufficient funds

}

return true

}

Thisfunctionevent ual lygives the cor rec t resu lt, but it has a nasty side effe ct. Whenanexces-

sive wit hdrawal isatt emp ted,the bal ance transientlydipsbelow zero. Thismay cause a con-

currentwit hdrawal for a modestsum tobespuriou sly rej e cte d. So ifBob tries tobuy a sports

car, Alice can’tpay for her mor ningcof fee.The pro blem isthat Withdraw is not at o mic:it

consists ofasequence ofthree sep arateoperat ions,eachofwhichacquires andthenreleases

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

SECTION 9.2. MUTUAL EXCLUSION: SYNC.MUTEX 265

themut exlock, but not hinglocks the whole sequence.

Ideally, Withdraw shouldacquirethe mut exlockoncearo und the whole operat ion. How ever,

this att emp t won’twor k:

// NOTE: incorrect!

func Withdraw(amount int) bool {

mu.Lock()

defer mu.Unlock()

Deposit(-amount)

if Balance() < 0 {

Deposit(amount)

return false // insufficient funds

}

return true

}

Deposit tr ies toacquirethe mut exlockasecon d time bycal ling mu.Lock(),but because

mu tex locksare not re-entrant—it’snot possibletolockamut exthat’salready locke d—this

le adstoadeadlockwhere not hingcan pro ceed,and Withdraw blocks forever.

Thereisagood reasonGo’smut exesare not re-entrant. Thepur pos e of a mut existoens ure

that certain invar iants ofthe shared var iables aremaintained atcriticalpointsdur ingprogram

exec ution.One ofthe invar iants is ‘‘no goroutine isaccessingthe shared var iables,’’ butthere

maybeaddition alinvar iants speciﬁctothe dat a st ruc tures thatthe mut exguard s.Whena

goro utine acquires a mut exlock, itmay assume thatthe invar iants hold. Whi le it holds the

lo ck, itmay updatethe shared var iables sothatthe invar iants aretemporar ily violate d.

Ho wever,whenitreleasesthe lock, itmustguarante e that order has beenrestore d andthe

invar iants holdonceagain. Alt hough a re-entrantmut exwou ldens ure thatnoother

goro utinesare accessingthe shared var iables, itcannot pro tec t theaddition alinvar iants of

thos e var iables.

Acommonsolut ion istodiv ide a functionsuchas Deposit into two:anunexp orted function,

deposit,thatassumesthe lockisalready heldand doesthe realwor k,and anexp orted func-

tion Deposit that acquires the lockbeforecal ling deposit.Wecan thenexpress Withdraw in

termsof deposit li kethis:

func Withdraw(amount int) bool {

mu.Lock()

defer mu.Unlock()

deposit(-amount)

if balance < 0 {

deposit(amount)

return false // insufficient funds

}

return true

}

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

266 CHAPTER 9. CONCURRENCY WITH SHARED VARIABLES

func Deposit(amount int) {

mu.Lock()

defer mu.Unlock()

deposit(amount)

}

func Balance() int {

mu.Lock()

defer mu.Unlock()

return balance

}

// This function requires that the lock be held.

func deposit(amount int) { balance += amount }

Of course,the deposit func tionshown hereissotrivialthatarealist ic Withdraw func tion

wouldn’tbot her cal lingit, but non ethelessitillustrates the princip le.

Encapsulat ion(§6.6), byreducingunexp ected int erac tions ina program, helps usmaintain

data str uctureinvar iants. For the samereason, encapsulat ionals o helps usmaintain conc ur-

renc y invar iants. Whenyou use a mut ex, makesurethatbot h it and the var iables itguard s are

notexp orted,whether the y arepackage-le vel var iables orthe ﬁelds ofastr uct.

9.3. Read/WriteMutexes: sync.RWMutex

In a ﬁtofanxiet y af ter seeinghis $100 depositvanish wit houtatrace,Bob writesaprogram to

ch eck his ban k balancehundredsoftimes a secon d.Herunsitathom e,atwor k,and onhis

ph one.The ban k notices thatthe incre asedtrafﬁcisdel aying dep osits andwit hdrawals,

becaus e al l the Balance re quests run sequential ly, holdingthe lockexc lusivelyand temporar-

ilypre venting other goroutinesfro m running.

Sincethe Balance func tiononlyneedsto re ad thestate ofthe var iable,itwou ldinfac t be safe

formultiple Balance callstorun conc urrently, solon g as no Deposit or Withdraw call isrun-

ning. Inthisscenar io we need a speci al kind oflockthatallowsread-onlyoperat ions to

proceed inparal lelwit h each other,but write operat ions tohavefullyexc lusiveaccess. This

lo ckiscal le d a mu ltipl e re aders, single writer lo ck, and in Go it’sprovide d by sync.RWMutex:

var mu sync.RWMutex

var balance int

func Balance() int {

mu.RLock() // readers lock

defer mu.RUnlock()

return balance

}

The Balance func tionnow cal lsthe RLock and RUnlock methodstoacquireand release a

re aders or sh are d lo ck. The Deposit func tion, whichisunchange d,cal lsthe mu.Lock and

mu.Unlock methodstoacquireand release a writ er or ex clu siv e lo ck.

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

SECTION 9.4. MEMORYSYNCHRONIZATION 267

Af ter thischange , most ofBob’s Balance re quests run in paral lelwit h each other and ﬁnish

more quickly. The lockisavai lable for moreofthe time, and Deposit re quests can pro ceed in

atimelymanner.

RLock canbeusedonlyifthere are nowritestoshare d var iables in the criticalsec tion.Ingen-

eral,weshouldnot assume that logically re ad-onlyfunctions ormet hodsdon’tals o up date

some var iables. For example, a met hod thatapp earstobeasimpleaccessormig htals o incre-

ment anint ernal usage count er, orupdateacache sothatrep eat cal lsare faster. Ifindou bt,

us e an exclusive Lock.

It’sonlyproﬁtabletouse an RWMutex when mostofthe goroutinesthatacquirethe lockare

re aders, andthe lockisunder cont ent ion,thatis, goroutinesroutinelyhavetowaittoacquire

it.An RWMutex re quires morecomplex internal bookkeeping , making itslowerthanaregu lar

mu tex for uncontended locks.

9.4. Memory Synchronization

Yo u maywon der why the Balance method needsmut ual exclusion, either channel-b ased or

mu tex-b ased.After all, unlike Deposit,itcon sists onlyofasingleoperat ion, sothere isno

dangerofanother goroutine exe cut ing ‘‘in the midd le’’ of it.There are two reasons we need a

mu tex. The ﬁrs t is thatit’sequ allyimp ortantthat Balance notexe cut e in the midd le of som e

ot her operat ionlike Withdraw.The secon d (andmoresubtle) reasonisthatsynchro nizat ion

is about morethanjustthe order ofexe cut ion of multiplegoroutines; synchronizat ionals o

af fec tsmemor y.

In a moder n comp uterthere may bedozensofpro cessors,eachwit h itsown local cache ofthe

main memor y.For efﬁciency,writestomemor y arebuf fered wit hin eachpro cessorand

ﬂushe d outtomain memor y on lywhennecessary.The y mayevenbecommitt edtomain

memory inadif ferentorder thanthe y were writt enbythe writing goro utine.Synchro nizat ion

pr imitiveslikechannel communic ations and mut exoperat ions cause the pro cessortoﬂush

outand commitall its accumulated writessothatthe effec tsofgoroutine exe cut ion up tothat

pointare guarante e d to bevisible togoroutinesrunningonother pro cessors.

Consider the possibleout putsofthe fol low ing snipp etofcode:

var x, y int

go func() {

x=1 // A1

fmt.Print("y:", y, " ") // A2

}()

go func() {

y=1 // B1

fmt.Print("x:", x, " ") // B2

}()

Sincethese two goro utinesare con cur rentand accessshare d var iables wit houtmut ual

exclusion, there isadat a race,soweshouldnot besur prisedthatthe program isnot

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

268 CHAPTER 9. CONCURRENCY WITH SHARED VARIABLES

deter minist ic. Wemig htexp ect ittoprint any one ofthese fourresults, whichcor respond to

intuit ive int erleavingsofthe lab ele d st atementsofthe program:

y:0 x:1

x:0 y:1

x:1 y:1

y:1 x:1

Thefourthlinecou ldbeexplained bythe sequence A1,B1,A2,B2 or by B1,A1,A2,B2,for

example. How ever, these two out com esmig htcom e as a sur prise:

x:0 y:0

y:0 x:0

butdep endingonthe compi ler,CPU,and manyother factors,the y canhappentoo.What

possible interleaving ofthe fourstatementscou ldexplain them?

Wi thin a singlegoroutine,the effec tsofeachstatement are guarante e d to occ ur in the order of

exec ution;goroutinesare sequ ent ial lyconsi stent.But inthe abs ence ofexplicitsynchro niza-

tion usingachannel ormut ex, there isnoguarante e that eventsare seeninthe sameorder by

al l goro utines. Alt hough goroutine A mu stobs erve the effec t of the write x=1before itreads

thevalue of y,itdoesnot necessarily obs erve the write to y done bygoroutine B,so A may

pr int a stale value of y.

It istempt ing totry tounderst and con cur rency asifitcor respond s to some interleaving ofthe

st atementsofeachgoroutine,but asthe exampleabove shows, thisisnot how a moder n com-

pi ler orCPU wor ks. Because the assig nmentand the Print refertodif ferentvar iables, a com-

pi ler may conclude thatthe order ofthe two statementscannot affec t theresult, andswap

them. Ifthe two goro utinesexe cut e on dif ferentCPUs, eachwit h itsown cach e, wr itesbyone

goro utine are not visible tothe other goroutine’s Print until the cachesare synchro nize d with

main memor y.

Al l thes e conc urrency pro blemscan beavoide d by the con sistent use ofsimple, est ablishe d

patt erns. Where possible, conﬁne var iables toasinglegoroutine;for all other variables, use

mu tualexc lusion.

9.5. Lazy Initialization: sync.Once

It isgood prac tice todefer an expensive initializat ionstepunt i l themom ent itisneeded.Ini-

tializingavar iable upfro ntincre asesthe start-uplatency ofaprogram andisunnecessary if

exec ution doesn’talways reach the par t of the program thatusesthatvar iable.Let’sretur n to

the icons var iable wesaw earlier in the chapt er:

var icons map[string]image.Image

Thisversionof Icon us es lazy ini tializati on:

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

SECTION 9.5. LAZY INITIALIZATION: SYNC.ONCE 269

func loadIcons() {

icons = map[string]image.Image{

"spades.png": loadIcon("spades.png"),

"hearts.png": loadIcon("hearts.png"),

"diamonds.png": loadIcon("diamonds.png"),

"clubs.png": loadIcon("clubs.png"),

}

// NOTE: not concurrency-safe!

func Icon(name string) image.Image {

if icons == nil {

loadIcons() // one-time initialization

}

return icons[name]

}

Fo r avar iable accessedbyonlyasinglegoroutine,wecan use the pattern above , butthispat-

tern isnot safeif Icon is cal le d conc urrently. Likethe ban k’soriginal Deposit func tion, Icon

consists ofmultiplesteps: ittests whether icons is nil,thenitloads the icons,thenitupdates

icons to a non-ni l value.Int uit ion mig htsug gestthatthe wor stpossibleout com e of the race

condit ion above isthatthe loadIcons func tioniscal le d several times. Whi le theﬁrs t

goro utine isbusyloadingthe icons,another goroutine ent ering Icon wouldﬁnd the var iable

st i l l equalto nil,and wou ldals o call loadIcons.

Butthisint uit ion isals o wron g.(We hop e that bynow you are developinganew int uit ion

ab out con cur rency,thatint uit ion s ab out con cur rency are not tobetrusted!) Recal l thedis-

cussionofmemor y from Sec tion 9.4. Inthe abs ence ofexplicitsynchro nizat ion, the compi ler

andCPU arefre e to reorder accessestomemor y in anynumberofways, solon g as the beh av-

iorofeachgoroutine issequential lycon sistent.One possiblereorder ingofthe statementsof

loadIcons is shown below.Itstoresthe emp tymap in the icons var iable beforepopu lat ingit:

func loadIcons() {

icons = make(map[string]image.Image)

icons["spades.png"] = loadIcon("spades.png")

icons["hearts.png"] = loadIcon("hearts.png")

icons["diamonds.png"] = loadIcon("diamonds.png")

icons["clubs.png"] = loadIcon("clubs.png")

}

Cons equently, a goroutine ﬁnding icons to benon-ni l maynot assume thatthe initializat ion

of the var iable iscomplete.

Thesimplestcor rec t way toens ure thatall goroutinesobs erve the effec tsof loadIcons is to

sy nchro nize themusingamut ex:

var mu sync.Mutex // guards icons

var icons map[string]image.Image

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

270 CHAPTER 9. CONCURRENCY WITH SHARED VARIABLES

// Concurrency-safe.

func Icon(name string) image.Image {

mu.Lock()

defer mu.Unlock()

if icons == nil {

loadIcons()

}

return icons[name]

}

Ho wever,the costofenforcingmut ual lyexc lusiveaccessto icons is thattwo goro utinescan-

notaccessthe var iable conc urrently, evenoncethe var iable has beensafelyinitialize d andwill

ne ver bemodiﬁed again. Thissug gests a multiple-readerslock:

var mu sync.RWMutex // guards icons

var icons map[string]image.Image

// Concurrency-safe.

func Icon(name string) image.Image {

mu.RLock()

if icons != nil {

icon := icons[name]

mu.RUnlock()

return icon

}

mu.RUnlock()

// acquire an exclusive lock

mu.Lock()

if icons == nil { // NOTE: must recheck for nil

loadIcons()

}

icon := icons[name]

mu.Unlock()

return icon

}

Thereare now two criticalsec tion s.The goroutine ﬁrs t acquires a reader lock,con sults the

map, thenreleasesthe lock. Ifanent rywas found (thecommoncas e), it isretur ned.Ifno

entr y was found,the goroutine acquires a writerlock. Thereisnoway toupg rade ashare d

lo cktoanexc lusiveone wit houtﬁrs t re leasingthe shared lock, sowemustreche ckthe icons

var iable in cas e anot her goroutine already initialize d it inthe int erim.

Thepattern above gives usgre atercon cur rency but iscomplex andthu s er ror-pro ne.

Fo rtunate ly, the sync packageprovides a speci alizedsolut ion tothe pro blem ofone-t imeini-

tializat ion: sync.Once.Con ceptu ally, a Once consists ofamut exand a boole an var iable that

re cords whether initializat ionhas taken place; the mut exguard s both theboole an andthe

client’sdat a st ruc tures. Thesolemet hod, Do,accepts the initializat ionfunctionasits argu-

ment.Let’suse Once to simplif y the Icon func tion:

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

SECTION 9.6. THE RACE DETECTOR 271

var loadIconsOnce sync.Once

var icons map[string]image.Image

// Concurrency-safe.

func Icon(name string) image.Image {

loadIconsOnce.Do(loadIcons)

return icons[name]

}

Each cal l to Do(loadIcons) lo cks the mut exand che cks the boole an var iable.Inthe ﬁrs t call,

in whichthe var iable isfalse, Do calls loadIcons andsets the var iable totrue. Subsequent

callsdonot hing, but the mut exsynchro nizat ionens uresthatthe effec tsof loadIcons on

memory (sp eciﬁcally, icons)becom e visibletoall goroutines. Using sync.Once in thisway,

we can avoidsharing var iables wit h ot her goroutinesunt i l they havebeenpro perly con-

st ruc ted.

Exercis e 9.2: Re writ e the PopCount examplefro m Section2.6.2 sothatitinitializes the looku p

tableusing sync.Once theﬁrs t time itisneeded.(Re alist ically, the costofsynchro nizat ion

wouldbepro hibit ive for a small and hig hly opt imize d func tionlike PopCount.)

9.6. The RaceDet ector

Ev enwit h thegre atest ofcare, it’sall too easy tomakecon cur rency mistakes. For tunately, the

Go runtime and toolchain areequip ped wit h asop histicate d andeasy-to-use dynamic analysis

to ol, the ra c e dete ctor.

Ju stadd the -race ﬂag toyour go build, go run,or go test command. Thiscausesthe com-

pi ler tobui ld amodiﬁed versionofyourapp lic ationortestwit h addition alins trument ation

that effe ctive lyrecords all accessestoshare d var iables thatocc urre d during exe cut ion,along

with theidentity ofthe goroutine thatreadorwro tethe var iable.Inaddition,the modiﬁed

prog ram re cords all synchro nizat ionevents, suchas go st atements, channel operat ions,and

callsto (*sync.Mutex).Lock, (*sync.WaitGroup).Wait,and soon. (Thecompleteset of

sy nchro nizat ioneventsisspeciﬁed bythe TheGoMemor y Mo d el do cumentthataccomp anies

thelangu agespeciﬁcation.)

Therace detec tor studies thisstreamofevents, look ing for cas es in whichone goroutine reads

or writesashare d var iable thatwas mostrecentlywritt enbyadif ferentgoroutine wit houtan

interveningsynchro nizat ionoperat ion. Thisindic ates a conc urrentaccesstothe shared var i-

able,and thu s adat a race.The toolprintsarep ort thatincludes the identity ofthe var iable,

andthe stacksofactivefunctioncal lsinthe readinggoroutine and the writing goro utine.This

is usuallysufﬁcienttopinpointthe pro blem. Sec tion 9.7 cont ainsanexampleofthe race

detec tor inaction.

Therace detec tor rep ortsall dat a races thatwereactuallyexe cut ed. How ever, itcan only

detec t race condit ion s that occ ur during a run; itcannot prove thatnon e wi l l ever occ ur.For

best results, makesurethatyourtests exercise yourpackages usingcon cur rency.

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

272 CHAPTER 9. CONCURRENCY WITH SHARED VARIABLES

Duetoext ra bookkeeping , aprogram bui ltwit h race detec tion needsmoretimeand memory

to run, but the overhe adistolerable evenfor manypro duc tionjobs. For inf requentlyocc ur-

ring race condit ion s,letting the race detec tor doits job can savehours ordays ofdebug ging.

9.7. Example: ConcurrentNon-Blocking Cache

In thissec tion,we’ll bui ld a concur rentnon-blocki n g cache, an abstrac tionthatsolves a

problem thatarisesoften in real-wor ldcon cur rentprogramsbut isnot wel l addressedby

exist ing librar ies. Thisisthe pro blem of memoizing afunction, thatis, cachingthe resultofa

func tionsothatitneed becompute d on lyonce. Our solut ion will becon cur rency-s afeand

wi l l avoidthe content ion ass oci ated wit h desig nsbas edonasinglelockfor the whole cache.

We’ll use the httpGetBody func tionbelow asanexampleofthe typ e of functionwemig ht

wanttomemoize.Itmakes an HTTP GETrequestand reads the requestbody. Cal lstothis

func tionare relative lyexp ensive , so we’dliketoavoid rep eat ingthemunnecessarily.

func httpGetBody(url string) (interface{}, error) {

resp, err := http.Get(url)

if err != nil {

return nil, err

}

defer resp.Body.Close()

return ioutil.ReadAll(resp.Body)

}

The ﬁnallinehides a minor subtlet y. ReadAll returnstwo results, a []byte andan error,but

since these are assig nable tothe declare d resu lttyp es of httpGetBody—interface{} and

error,respectively—we can retur n theresultofthe cal l withoutfur therado.Wechose this

return typ e for httpGetBody so thatitconformstothe typ e of functions thatour cache is

desig ned tomemoize.

Here’sthe ﬁrs t draf t of the cache:

gopl.io/ch9/memo1

// Package memo provides a concurrency-unsafe

// memoization of a function of type Func.

package memo

// A Memo caches the results of calling a Func.

type Memo struct {

fFunc

cache map[string]result

}

// Func is the type of the function to memoize.

type Func func(key string) (interface{}, error)

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

SECTION 9.7. EXAMPLE: CONCURRENT NON-BLOCKING CACHE 273

type result struct {

value interface{}

err error

}

func New(f Func) *Memo {

return &Memo{f: f, cache: make(map[string]result)}

}

// NOTE: not concurrency-safe!

func (memo *Memo) Get(key string) (interface{}, error) {

res, ok := memo.cache[key]

if !ok {

res.value, res.err = memo.f(key)

memo.cache[key] = res

}

return res.value, res.err

}

A Memo inst anceholds the function f to memoize,oftyp e Func,and the cache,whichisa

mappingfro m st rings to results. Each result is simply the pair ofresults retur ned byacal l

to f—avalue and an error.We’ll showseveral variation s of Memo as the desig n prog resses, but

al l wi l l sh are these basic aspects.

An exampleofhow touse Memo appearsbelow.For eachelementinastreamofincoming

URLs, wecal l Get,log gingthe latency ofthe cal l andthe amountofdat a it retur ns:

m:=memo.New(httpGetBody)

for url := range incomingURLs() {

start := time.Now()

value, err := m.Get(url)

if err != nil {

log.Print(err)

}

fmt.Printf("%s, %s, %d bytes\n",

url, time.Since(start), len(value.([]byte)))

}

We can use the testing package(thetopic ofChapt er11) tosystematicallyinv est igatethe

ef fec t of memoizat ion. Fro m thetestout put below,wesee thatthe URL streamcontains

dup lic ates, andthatalt hough the ﬁrs t call to (*Memo).Get foreachURL takes hundredsof

mi l lisecond s,the secon d re questretur nsthe sameamountofdat a in under a millis econd.

$gotest -v gopl.io/ch9/memo1

=== RUN Test

https://golang.org, 175.026418ms, 7537 bytes

https://godoc.org, 172.686825ms, 6878 bytes

https://play.golang.org, 115.762377ms, 5767 bytes

http://gopl.io, 749.887242ms, 2856 bytes

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

274 CHAPTER 9. CONCURRENCY WITH SHARED VARIABLES

https://golang.org, 721ns, 7537 bytes

https://godoc.org, 152ns, 6878 bytes

https://play.golang.org, 205ns, 5767 bytes

http://gopl.io, 326ns, 2856 bytes

--- PASS: Test (1.21s)

PASS

ok gopl.io/ch9/memo1 1.257s

Thistestexe cut esall cal lsto Get sequential ly.

SinceHTTPrequests areagre atopp ortunity for paral lelism, let’schange the testsothatit

makesall requests conc urrently. The testusesasync.WaitGroup to waitunt i l thelastrequest

is completebeforeretur ning.

m:=memo.New(httpGetBody)

var n sync.WaitGroup

for url := range incomingURLs() {

n.Add(1)

go func(url string) {

start := time.Now()

value, err := m.Get(url)

if err != nil {

log.Print(err)

}

fmt.Printf("%s, %s, %d bytes\n",

url, time.Since(start), len(value.([]byte)))

n.Done()

}(url)

}

n.Wait()

Thetestrunsmuchfaster, but unfor tunatelyitisunlikelytowor k correc tly all the time. We

maynot ice unexp ected cache misses, orcache hitsthatretur n incorrec t values, oreven

crashes.

Wo rse,itislikelytowor k correc tly some of the time, sowemay not evennot ice thatithas a

problem. But ifwerun itwit h the -race ﬂag , therace detec tor (§9.6) often printsarep ort

such asthisone:

$gotest -run=TestConcurrent -race -v gopl.io/ch9/memo1

=== RUN TestConcurrent

...

WARNING: DATA RACE

Write by goroutine 36:

runtime.mapassign1()

~/go/src/runtime/hashmap.go:411 +0x0

gopl.io/ch9/memo1.(*Memo).Get()

~/gobook2/src/gopl.io/ch9/memo1/memo.go:32 +0x205

...

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

SECTION 9.7. EXAMPLE: CONCURRENT NON-BLOCKING CACHE 275

Previous write by goroutine 35:

runtime.mapassign1()

~/go/src/runtime/hashmap.go:411 +0x0

gopl.io/ch9/memo1.(*Memo).Get()

~/gobook2/src/gopl.io/ch9/memo1/memo.go:32 +0x205

...

Found 1 data race(s)

FAIL gopl.io/ch9/memo1 2.393s

Thereference to memo.go:32 te lls us thattwo goro utineshaveupdated the cache map

withoutany int erveningsynchro nizat ion. Get is not con cur rency-s afe: ithas a dat a race.

28 func (memo *Memo) Get(key string) (interface{}, error) {

29 res, ok := memo.cache[key]

30 if !ok {

31 res.value, res.err = memo.f(key)

32 memo.cache[key] =res

33 }

34 return res.value, res.err

35 }

Thesimplestway tomakethe cache con cur rency-s afeistouse monit or-b ased synchro niza-

tion.All weneed to do is add a mut extothe Memo,acquirethe mut exlock at the start of Get,

andrelease itbefore Get returns, sothatthe two cache op erat ions occ ur within the critical

section:

gopl.io/ch9/memo2

type Memo struct {

fFunc

mu sync.Mutex // guards cache

cache map[string]result

}

// Get is concurrency-safe.

func (memo *Memo) Get(key string) (value interface{}, err error) {

memo.mu.Lock()

res, ok := memo.cache[key]

if !ok {

res.value, res.err = memo.f(key)

memo.cache[key] = res

}

memo.mu.Unlock()

return res.value, res.err

}

No w therace detec tor issilent, evenwhenrunningthe tests conc urrently. Unfor tunatelythis

ch ange to Memo re versesour earlier per for mance gains. Byholdingthe lockfor the durat ionof

each cal l to f, Get serializes allthe I/O operat ions we int ended toparal lelize.Whatweneed is

a non-blocki n g cach e, on e that does not ser ialize cal lstothe functionitmemoizes.

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

276 CHAPTER 9. CONCURRENCY WITH SHARED VARIABLES

In the next implementation of Get,below,the cal linggoroutine acquires the locktwice: once

forthe looku p,and thenasecon d time for the updateifthe looku p returned not hing. In

between, other goroutinesare fre e to use the cache.

gopl.io/ch9/memo3

func (memo *Memo) Get(key string) (value interface{}, err error) {

memo.mu.Lock()

res, ok := memo.cache[key]

memo.mu.Unlock()

if !ok {

res.value, res.err = memo.f(key)

// Between the two critical sections, several goroutines

// may race to compute f(key) and update the map.

memo.mu.Lock()

memo.cache[key] = res

memo.mu.Unlock()

}

return res.value, res.err

}

Theper for mance improv esagain, but now wenot ice thatsom e URLs arebeingfetch edtwice.

Thishappens whentwo ormoregoroutinescal l Get forthe sameURL at aboutthe sametime.

Both con sultthe cache,ﬁnd no value there , andthencal l theslowfunction f.Thenbot h of

them updatethe map wit h theresultthe y obtained.One ofthe results isoverwritt enbythe

ot her.

Ideallywe’dliketoavoid thisredundant wor k.Thisfeature issom etimescal le d du p licate

sup pressi on.Inthe versionof Memo below, eachmap elementisapoint ertoan entry st ruc t.

Each entry cont ainsthe memoize d resu ltofacal l to the function f,asbefore, but it addition-

al lycontainsachannel cal le d ready.Justafter the entry’s result hasbeenset, thischannel

wi l l be clos ed, to broadcast (§8.9) toany other goroutinesthatitisnow safefor themtoread

theresultfro m the entry.

gopl.io/ch9/memo4

type entry struct {

res result

ready chan struct{} // closed when res is ready

}

func New(f Func) *Memo {

return &Memo{f: f, cache: make(map[string]*entry)}

}

type Memo struct {

fFunc

mu sync.Mutex // guards cache

cache map[string]*entry

}

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

SECTION 9.7. EXAMPLE: CONCURRENT NON-BLOCKING CACHE 277

func (memo *Memo) Get(key string) (value interface{}, err error) {

memo.mu.Lock()

e:=memo.cache[key]

if e == nil {

// This is the first request for this key.

// This goroutine becomes responsible for computing

// the value and broadcasting the ready condition.

e=&entry{ready: make(chan struct{})}

memo.cache[key] = e

memo.mu.Unlock()

e.res.value, e.res.err = memo.f(key)

close(e.ready) // broadcast ready condition

}else {

// This is a repeat request for this key.

memo.mu.Unlock()

<-e.ready // wait for ready condition

}

return e.res.value, e.res.err

}

Acal l to Get nowinv olves acquir ingthe mut exlockthatguard s the cache map, looking inthe

mapfor a point er to an exist ing entry,allocat ingand ins ertinganew entry if non e was

found, thenreleasingthe lock. Ifthere was an exist ing entry,its value isnot necessarily ready

yet—anot her goroutine cou ldstill becal lingthe slow function f—so the cal linggoroutine

mu stwaitfor the entry’s ‘‘re ady’’ condit ion beforeitreads the entry’s result.Itdoesthisby

re adingavalue fro m the ready ch annel,since thisoperat ionblo cks until the channel isclos ed.

If there was noexist ing entry,thenbyins ertinganew ‘‘notready’’ entry into the map,the

currentgoroutine becom esresponsible for inv oking the slow function, updat ingthe entry,

andbro adc asting the readinessofthe new entry to any other goroutinesthatmig ht(by then)

be waiting for it.

No tice thatthe var iables e.res.value and e.res.err in the entry areshare d amon g

mu ltiplegoroutines. Thegoroutine thatcre atesthe entry sets their values, andother

goro utinesreadtheir values oncethe ‘‘re ady’’ condit ion has beenbro adc ast. Despit e being

accessedbymultiplegoroutines, nomut exlockisnecessary.The closingofthe ready ch annel

happ ens before anyother goroutine receivesthe bro adc astevent,sothe write to those var iables

in the ﬁrs t goro utine happ ens before they are read bysubsequentgoroutines. Thereisnodat a

race.

Ourcon cur rent, dup lic ate-sup pressing, non-blo cking cache iscomplete.

Theimp lementation of Memo ab ove usesamut extoguard a map variablethatisshare d by

each goroutine thatcal ls Get.It’sint erest ing tocontrastthisdesig n with analt ernat ive one in

whichthe map variableisconﬁne d to a moni tor goroutine to whichcal lersof Get mu stsenda

mess age .

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

278 CHAPTER 9. CONCURRENCY WITH SHARED VARIABLES

Thedeclarat ions of Func, result,and entry remain as before:

// Func is the type of the function to memoize.

type Func func(key string) (interface{}, error)

// A result is the result of calling a Func.

type result struct {

value interface{}

err error

}

type entry struct {

res result

ready chan struct{} // closed when res is ready

}

Ho wever,the Memo type now con sists ofachannel, requests,through whichthe cal ler of Get

communic ates wit h themonit or goro utine.The elementtyp e of the channel isarequest.

Usingthisstr ucture, the cal ler of Get send s themonit or goro utine bot h thekey,thatis, the

argumenttothe memoize d func tion, andanother channel, response,overwhichthe result

shouldbesentbackwhenitbecom esavai lable.Thischannel will car ryonlyasinglevalue.

gopl.io/ch9/memo5

// A request is a message requesting that the Func be applied to key.

type request struct {

key string

response chan<- result // the client wants a single result

}

type Memo struct{ requests chan request }

// New returns a memoization of f. Clients must subsequently call Close.

func New(f Func) *Memo {

memo := &Memo{requests: make(chan request)}

go memo.server(f)

return memo

}

func (memo *Memo) Get(key string) (interface{}, error) {

response := make(chan result)

memo.requests <- request{key, response}

res := <-response

return res.value, res.err

}

func (memo *Memo) Close() { close(memo.requests) }

The Get method,above , createsarespons e ch annel,putsitinthe request, sends ittothe moni-

torgoroutine,thenimmediate lyreceivesfro m it.

The cache var iable isconﬁne d to the monitorgoroutine (*Memo).server,shown below.The

monitorreads requests in a loop unt i l therequestchannel isclos edbythe Close method.For

each request, itcon sults the cache,cre ating and ins ertinganew entry if non e was found.

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

SECTION 9.7. EXAMPLE: CONCURRENT NON-BLOCKING CACHE 279

func (memo *Memo) server(f Func) {

cache := make(map[string]*entry)

for req := range memo.requests {

e:=cache[req.key]

if e == nil {

// This is the first request for this key.

e=&entry{ready: make(chan struct{})}

cache[req.key] = e

go e.call(f, req.key) // call f(key)

}

go e.deliver(req.response)

}

func (e *entry) call(f Func, key string) {

// Evaluate the function.

e.res.value, e.res.err = f(key)

// Broadcast the ready condition.

close(e.ready)

}

func (e *entry) deliver(response chan<- result) {

// Wait for the ready condition.

<-e.ready

// Send the result to the client.

response <- e.res

}

In a simi lar manner tothe mut ex-b ased version, the ﬁrs t re questfor a given key becom es

resp onsible for cal lingthe function f on thatkey,storing the resultinthe entry,and bro ad-

cast ing the readinessofthe entry by closingthe ready ch annel.Thisisdon e by

(*entry).call.

Asubsequentrequestfor the samekey ﬁnd s theexist ing entry in the map,waits for the result

to becom e re ady,and sends the resultthrough the respons e ch annel tothe clientgoroutine

that cal le d Get.Thisisdon e by (*entry).deliver.The call and deliver methodsmustbe

called intheir own goroutinestoens ure thatthe monitorgoroutine doesnot stoppro cessing

ne w re quests.

Thisexampleshows thatit’spossibletobui ld many con cur rentstr uctures usingeit her ofthe

twoappro ach es—sh are d var iables andlocks, orcommunic atingsequential pro cesses—

withoutexcessive complexity.

It’snot always obv iou s whichappro ach ispreferable in a given situ ation,but it’swor th

know ing how the y correspond.Som etimesswitchingfro m on e approach to the other can

make yourcodesimpler.

Exercis e 9.3: Extend the Func type and the (*Memo).Get method sothatcal lersmay provide

an opt ion al done ch annel through whichthe y cancancelthe operat ion(§8.9). Theresults ofa

cancel led Func call shouldnot becache d.

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

280 CHAPTER 9. CONCURRENCY WITH SHARED VARIABLES

9.8. Goroutines and Threads

In the pre vious chapt erwesaid thatthe dif ference bet weengoroutinesand operat ingsystem

(OS) threads cou ldbeignored unt i l later. Alt hough the dif ferences bet weenthemare ess en-

tial lyquant itative , abig enough quant itative dif ference becom esaqualitative one,and soitis

with goroutinesand threads.The timehas now com e to distinguish them.

9.8.1. Gro wable Stacks

Each OSthreadhas a ﬁxe d-size blo ckofmemor y (often as large as2MB) for its sta ck,the wor k

area where itsaves the local variables offunctioncal lsthatare inprogressortemporar ily

su spende d whileanother functioniscal le d.This ﬁxe d-size stack issimultane ously too much

andtoo lit tle.A2MB stack wou ldbeahugewaste ofmemor y foralitt legoroutine,suchas

on e that merelywaits for a WaitGroup then clos es achannel.It’snot uncommonfor a Go

prog ram to cre ate hundredsofthous and s of goroutinesatone time, whichwou ldbeimp ossi-

blewit h st acksthislarge . Ye tdespit e their size, ﬁxe d-size stacksare not always big enough for

themostcomplex anddeeply rec ursiveoffunctions.Chang ing the ﬁxe d size can improv e

sp ace efﬁciency and allow morethreads tobecre ate d, or it can enable moredeeply rec ursive

func tions,but it cannot dobot h.

In contrast, a goroutine startslifewit h asmall stack,typic ally 2KB.Agoro utine’sstack,like

thestack ofanOSthread, holds the local variables ofactiveand suspended functioncal ls, but

un li keanOSthread, a goroutine’sstack isnot ﬁxe d;itgrows andshr inks as needed.The size

limitfor a goroutine stack may beasmuchas1GB,ordersofmag nitude largerthanatypic al

ﬁxe d-size threadstack,thoug h of cours e fe w goro utinesuse thatmuch.

Exercis e 9.4: Cons truct a pip elinethatconne cts an arbit rar y numb erofgoroutineswit h ch an-

nel s.Whatisthe maximum numberofpip elinestagesyou can create wit houtrunningout of

memory?How lon g do es avalue taketotransitthe ent ire pip eline?

9.8.2. Goroutine Scheduling

OS threads are sch edu led bythe OSker nel.Every few millis econd s,ahardwaretimer inter-

ruptsthe pro cessor, whichcausesaker nel functioncal le d the sche dul er to beinv oke d.This

func tionsuspend s thecur rentlyexe cut ing threadand saves its reg istersinmemor y,looks over

thelistofthreads and decides whichone shouldrun next, restoresthatthread’sreg istersfro m

memory,thenresumes the exe cut ion of thatthread. Because OSthreads are sch edu led bythe

kernel,passingcontrol fro m on e thre adtoanother requires a full cont ext switch,thatis, sav ing

thestate ofone userthreadtomemor y,restoring the state ofanother,and updat ingthe

sche duler’sdat a st ruc tures. Thisoperat ionisslow, due toits poorlocality and the numberof

memory accessesrequired,and has histor icallyonlygot ten wor seasthe numberofCPU cyc les

re quired toaccessmemor y hasincre ased.

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

SECTION 9.8. GOROUTINES AND THREADS 281

TheGoruntime containsits own sch edu ler thatusesatechnique known as m:n sch edu ling,

becaus e it multiplexes (orsch edu les) m goro utineson n OS threads.The job ofthe Go

sche duler isanalogou s to thatofthe ker nel sch edu ler,but it iscon cer ned onlywit h the

goro utinesofasingleGoprogram.

Un likethe operat ingsystem’sthreadsch edu ler,the Gosch edu ler isnot inv oke d periodic ally

by a hardwaretimer,but imp licitlybycer tain Golangu agecon str ucts. For example, whena

goro utine cal ls time.Sleep or blo cks in a channel ormut exoperat ion, the sch edu ler putsitto

sleepand runsanother goroutine unt i l it istimetowakethe ﬁrs t on e up.Because itdoesn’t

ne e d aswitch to ker nel context, res che dulingagoroutine ismuchche aperthanres che dulinga

thre ad.

Exercis e 9.5: Wr ite a program wit h twogoroutinesthatsendmessagesbackand for thover

twounbuf fered channel s in ping-p ong fashion. How manycommunic ations per secon d can

theprogram sustain?

9.8.3. GOMAXPROCS

TheGosch edu ler usesaparameter cal le d GOMAXPROCS to deter minehow many OS threads

maybeactivelyexe cut ing Gocodesimultane ously.Its default value isthe numberofCPUson

themachine, soonamachinewit h 8CPUs, the sch edu ler will sch edu leGocodeonupto8OS

thre ads atonce. (GOMAXPROCS is the n in m:n sche duling.)Goroutinesthatare sle epingor

blocke d in a communic ation do not need a thread at all.Goroutinesthatare blo cke d in I/O or

ot her systemcal lsorare cal lingnon-G o func tions,doneed anOSthread, but GOMAXPROCS

ne e d notaccountfor them.

Yo u canexplicitlycontrol thisparameter usingthe GOMAXPROCS enviro nment var iable orthe

runtime.GOMAXPROCS func tion. Wecan see the effec t of GOMAXPROCS on thislit tle program,

whichprints an end lessstreamofzeros andones:

for {

go fmt.Print(0)

fmt.Print(1)

}

$GOMAXPROCS=1 go run hacker-cliché.go

111111111111111111110000000000000000000011111...

$GOMAXPROCS=2 go run hacker-cliché.go

010101010101010101011001100101011010010100110...

In the ﬁrs t run, at mostone goroutine was exec ute d at a time. Initial ly, itwas the main

goro utine,whichprintsones. After a per iodoftime, the Gosch edu ler put it tosle ep andwoke

up the goroutine thatprintszeros, giv ing itatur n to run onthe OSthread. Inthe secon d run,

thereweretwo OSthreads avai lable,sobot h goro utinesran simultane ously,print ing dig its at

ab out the samerate. Wemuststressthatmanyfac tor s areinv olved ingoroutine sch edu ling,

andthe runtime iscon stant lyevo l ving, soyourresults may differfro m theonesabove .

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

282 CHAPTER 9. CONCURRENCY WITH SHARED VARIABLES

Exercis e 9.6: Me asure how the per for mance ofacompute-b oundparal lelprogram (see Exer-

cise 8.5) varies wit h GOMAXPROCS.Whatisthe opt imalvalue onyourcomputer? How many

CPUsdoesyourcomputerhave?

9.8.4. Goroutines HaveNoIdentity

In mostoperat ingsystems and programminglangu ages thatsup por t mu ltithre ading, the cur-

rent threadhas a dist inc t identity thatcan beeasi lyobt ained asanordinar y value,typic ally an

integerorpoint er. Thismakes iteasy tobui ld an abstrac tioncal le d thre ad-local storage,which

is ess ent ial lyaglobalmap keyed bythreadidentity,sothateachthreadcan store and ret rie ve

values indep endentofother threads.

Goro utineshavenonot ion of identity thatisaccessibletothe programmer.Thisisbydesig n,

since thread-lo cal storagetends tobeabu sed.For example, inaweb ser ver implemente d in a

language wit h thre ad-lo cal storage, it’scommonfor manyfunctions toﬁnd infor mat ionabout

theHTTPrequestonwhose beh alf the y arecur rentlywor kingbylooking inthatstorage.

Ho wever,justaswit h prog ramsthatrelyexcessive lyonglobalvar iables, thiscan lead toan

un healt hy ‘‘ac tionatadistance’’ in whichthe beh avior of a functionisnot deter mined byits

arguments alone,but bythe identity ofthe threadinwhichitruns. Con sequently, ifthe iden-

tity ofthe threadshouldchange—some wor ker threads are enliste d to help, say—thefunction

misb ehavesmysteriou sly.

Go encourages a simpler sty leofprogramminginwhichparametersthataffec t thebeh avior of

afunctionare explicit. Not onlydoesthismakeprogramseasier toread, but it lets usfre ely

assig n su btasksofagiven functiontomanydif ferentgoroutineswit houtwor rying about their

identity.

Yo u ’ve now learned about all the langu agefeaturesyou need for writing Goprograms. Inthe

next two chapt ers,we’ll stepbacktolookatsom e of the prac tices andtools thatsup por t

prog ramminginthe large:how tostr uctureapro jec t as a set ofpackages, andhow toobt ain,

build, test,benchmark,proﬁle,document, andshare those packages.

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

Packages and the GoTool

Amodest-size program today might contain 10,000 functions.Yet its author need thin k ab out

on lyafew ofthemand desig n even fewer,because the vastmaj ority werewritt enbyothers

andmade availablefor reuse through pack ages.

Go com eswit h ov er100 stand ard packages thatprovide the found ation s formostapp lic ations.

TheGocommunity,athr iving ecosystemofpackagedesig n, sh aring , reus e,and imp rov ement,

haspublishe d many more, and you can ﬁnd a searc hable index ofthemat http://godoc.org.

In thischapt er, we’ll showhow touse exist ing packages andcre ate new ones.

Go als o comeswit h the go to ol, a sop histicate d butsimple-to-use command for manag ing

worksp aces ofGopackages. Since the beg inningofthe book, we’ve beenshowing how touse

the go to oltodow nlo ad,bui ld,and run exampleprograms. Inthischapt er, we’ll lookatthe

to ol’sunderly ing con cepts andtourmoreofits cap abilit ies, whichinclude print ing documen-

tation and quer yingmet adat a ab out the packages in the wor ksp ace.Inthe next chapt erwe’ll

explore its testing features.

10.1. Introduction

Thepur pos e of any packagesystemistomakethe desig n andmaintenanceoflarge programs

prac tic al by gro uping rel ate d fe aturestoget her into units thatcan beeasi lyunderstood and

ch ange d,indep endentofthe other packages ofthe program. This mo dul arity al lows packages

to beshare d andreusedbydif ferentpro jec ts, dist ribut edwit hin an organizat ion, ormade

avai lable tothe wider wor ld.

Each packagedeﬁnesadistinc t name space thatenclos es itsidentiﬁers. Eachnameisass oci-

ated wit h apar tic ularpackage, letting uscho ose short,cle ar namesfor the typ es, func tions,

andsoonthatweuse mostoften, wit houtcre ating conﬂic ts with other par ts of the program.

283

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

284 CHAPTER 10. PACKAGES AND THE GO TOOL

Packages also provide encapsul ati on by control lingwhichnames arevisible orexp orted

outside the package. Restr ictingthe visibilit y of packagemembers hides the helperfunctions

andtyp es behindthe package’sAPI,allow ing the packagemaintainer tochange the imp lemen-

tation wit h conﬁdence thatnocodeoutside the packagewill beaffec ted.Restr ictingvisibilit y

also hides variables sothatclients can accessand updatethemonlythrough exp orted func-

tion s that preserveint ernal invar iants orenforce mut ual exclusion in a con cur rentprogram.

Wh enwechange a ﬁle,wemustrecompi lethe ﬁle’spackageand pot ent ial lyall the packages

that dependonit. Gocompi lat ionisnot ably fasterthanmostother comp ile d languages, even

when bui ldingfro m scratch.There are three main reasons for the compi ler’sspeed.First,all

importsmustbeexplicitlyliste d at the beg inningofeachsource ﬁle,sothe compi ler does not

have toreadand pro cessanent ire ﬁle todeter mineits dependencies. Secon d,the dep enden-

cies ofapackagefor m adirec ted acyclic graph, andbecause there are nocyc les, packages can

be compi led sep aratelyand perhaps in paral lel. Final ly, the obj e ctﬁle for a compi led Gopack-

agerecords exp ort infor mat ionnot justfor the packageits elf,but for itsdep endencies too.

Wh encompi lingapackage, the compi ler mustreadone obj e ctﬁle for eachimp ort but need

notlookbeyon d thes e ﬁles.

10.2. Import Paths

Each packageisidentiﬁed byaunique str ing cal le d its import pat h.Imp ort pat hsare the

st rings thatapp ear in import de clarat ions.

import (

"fmt"

"math/rand"

"encoding/json"

"golang.org/x/net/html"

"github.com/go-sql-driver/mysql"

)

As wemention edinSec tion 2.6.1, the Golangu agespeciﬁcation doesn’tdeﬁne the meaningof

thes e st rings orhow todeter mineapackage’simp ort pat h, butleavesthese issues tothe tools.

In thischapt er, we’ll takeadet aile d lo okathow the go to olint erprets them, since that’swhat

themaj ority ofGoprogrammersuse for bui lding, testing , andsoon. Other tools doexist,

though.For example, GoprogrammersusingGoogle’sint ernal multi-l angu agebui ld system

fo llowdif ferentrules for namingand locat ingpackages, specif yingtests, andsoon, thatmore

clos ely match the convent ion s of thatsystem.

Fo r packages you int end toshare orpublish,imp ort pat hsshouldbegloballyunique.Toavoid

conﬂic ts, theimp ort pat hsofall packages other thanthose fro m thestand ard librar y should

st art wit h theInt ernet domain nameofthe organizat ionthatownsorhosts the package; this

also makes itpossibletoﬁnd packages. For example, the declarat ionabove imp ortsanHTML

pars ermaintained bythe Goteamand a popu lar third-p arty MySQLdat abas e dr iver.

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

SECTION 10.4. IMPORTDECLARATIONS 285

10.3. The PackageDeclaration

A package de clarat ionisrequired atthe start ofevery Gosource ﬁle.Its main pur pos e is to

deter minethe defau ltidentiﬁer for thatpackage(called the pack age nam e)whenitisimp orted

by another package.

Fo r example, every ﬁle ofthe math/rand packagestartswit h package rand,sowhenyou

import thispackage, you can accessits members as rand.Int, rand.Float64,and soon.

package main

import (

"fmt"

"math/rand"

)

func main() {

fmt.Println(rand.Int())

}

Conv ent ion ally, the packagenameisthe lastseg mentofthe imp ort pat h, andasaresult, two

packages may havethe samenameeventhoug h their import pat hsnecessarily dif fer.For

example, the packages whose imp ort pat hsare math/rand and crypto/rand both havethe

name rand.We’ll see how touse bot h in the sameprogram in a mom ent.

Thereare three maj orexception s to the ‘‘last seg ment’’ conv ent ion.The ﬁrs t is thatapackage

deﬁningacommand (an exec utableGoprogram) always has the name main,regardlessofthe

package’simp ort pat h. Thisisasig nal to go build (§10.7.3) thatitmustinv oke the lin ker to

make anexe cut able ﬁle.

Thesecon d exception isthatsom e ﬁles in the direc tor y mayhavethe sufﬁx _test on their

packagenameifthe ﬁle nameend s with _test.go.Suchadirec tor y maydeﬁne two packages:

theusu alone,plu s anot her one cal le d an exter nal testpackage.The _test sufﬁxsig nalsto

go test that itmustbui ld both packages, anditindic ates which ﬁles belon g to eachpackage.

External testpackages areusedtoavoid cyc les in the imp ort grapharisingfro m dep endencies

of the test; the y arecov ere d in moredet ail in Sec tion 11.2.4.

Thethirdexception isthatsom e to ols for dep endency managementapp end versionnumber

sufﬁxestopackageimp ort pat hs, suchas "gopkg.in/yaml.v2".The packagenameexc ludes

thesufﬁx, so in thiscas e it wou ldbejust yaml.

10.4. Import Declarations

AGosourceﬁle may cont ain zeroormore import de clarat ions immediate lyafter the package

de clarat ionand beforethe ﬁrs t non-import declarat ion. Eachimp ort declarat ionmay specif y

theimp ort pat h of a singlepackage, ormultiplepackages in a parenthesize d list.The two

formsbelow are equivalentbut the secon d form ismorecommon.

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

286 CHAPTER 10. PACKAGES AND THE GO TOOL

import "fmt"

import "os"

import (

"fmt"

"os"

)

Imported packages may begro upedbyint roducingblank lines; suchgro upingsusu allyindi-

cate dif ferentdom ains. Theorder isnot sig niﬁcant,but byconvent ion the lines ofeachgro up

aresor ted alp hab etic ally.(Both gofmt and goimports wi l l groupand sor t foryou .)

import (

"fmt"

"html/template"

"os"

"golang.org/x/net/html"

"golang.org/x/net/ipv4"

)

If weneed toimp ort two packages whose names arethe same, like math/rand and

crypto/rand,int o athirdpackage, the imp ort declarat ionmustspecif y an alternat ive name

for at least one ofthemtoavoid a conﬂic t. Thisiscal le d a renami n g import.

import (

"crypto/rand"

mrand "math/rand" // alternative name mrand avoids conflict

)

Thealt ernat ive nameaffec tsonlythe imp orting ﬁle.Other ﬁles, evenonesinthe samepack-

age, may import the packageusingits default name, oradif ferentname.

Arenamingimp ort may beusefulevenwhenthere isnoconﬂic t. If the nameofthe imp orted

packageisunw ieldy,asissom etimesthe cas e foraut omat ical lygenerated code, anabbre viate d

name may bemoreconvenient. Thesameshort nameshouldbeusedcon sistent lytoavoid

conf usion. Cho osinganalt ernat ive namecan helpavoid conﬂic ts with commonlocal variable

names. For example, inaﬁle wit h many local variables named path,wemig htimp ort the

st and ard "path" packageas pathpkg.

Each imp ort declarat ionest ablishesadep endency fro m thecur rentpackagetothe imp orted

package. The go build to olrep orts an error ifthese dep endencies for m acyc le.

10.5. Blank Imports

It isanerror toimp ort a packageint o aﬁle but not refer tothe nameitdeﬁneswit hin that ﬁle.

Ho wever,onocc asionwemustimp ort a packagemerelyfor the side effe cts ofdoing so: evalu-

at ionofthe initializer expressions ofits package-le vel var iables andexe cut ion of its init func-

tion s (§2.6.2). Tosup pressthe ‘‘unus edimp ort’’ er ror wewou ldother wis e encounter, wemust

us e arenamingimp ort inwhichthe alt ernat ive nameis _,the blank identiﬁer.Asusu al, the

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

SECTION 10.5. BLANK IMPORTS 287

bl ank identiﬁer can never bereference d.

import _ "image/png" // register PNG decoder

Thisisknown asabl ank imp ort.Itismostoften usedtoimp lementacompi le-t ime

me chanism where bythe main program can enable opt ion alfeaturesbyblank-importingaddi-

tion alpackages. First we’ll see how touse it, thenwe’ll see how itwor ks.

Thestand ard librar y’s image packageexp ortsaDecode func tionthatreads bytes fro m an

io.Reader, ﬁgures out whichimage for mat was usedtoencodethe dat a, invo kes the

appropriatedecoder,thenretur nsthe resulting image.Image.Using image.Decode,it’seasy

to bui ld asimpleimage converter thatreads animage inone for mat andwritesitout in

anot her :

gopl.io/ch10/jpeg

// The jpeg command reads a PNG image from the standard input

// and writes it as a JPEG image to the standard output.

package main

import (

"fmt"

"image"

"image/jpeg"

_"image/png" // register PNG decoder

"io"

"os"

)

func main() {

if err := toJPEG(os.Stdin, os.Stdout); err != nil {

fmt.Fprintf(os.Stderr, "jpeg: %v\n", err)

os.Exit(1)

}

func toJPEG(in io.Reader, out io.Writer) error {

img, kind, err := image.Decode(in)

if err != nil {

return err

}

fmt.Fprintln(os.Stderr, "Input format =", kind)

return jpeg.Encode(out, img, &jpeg.Options{Quality: 95})

}

If wefeed the out put of gopl.io/ch3/mandelbrot (§3.3) tothe converter program, itdetec ts

thePNG inp utfor mat andwritesaJPEGversionofFigure3.3.

$gobuild gopl.io/ch3/mandelbrot

$gobuild gopl.io/ch10/jpeg

$./mandelbrot | ./jpeg >mandelbrot.jpg

Input format = png

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

288 CHAPTER 10. PACKAGES AND THE GO TOOL

No tice the blank imp ort of image/png.Wit houtthatline, the program comp iles andlin ks as

usualbut can nolon g errecog nize or decodeinp ut in PNG for mat:

$gobuild gopl.io/ch10/jpeg

$./mandelbrot | ./jpeg >mandelbrot.jpg

jpeg: image: unknown format

Here’show itwor ks. Thestand ard librar y prov ides deco dersfor GIF,PNG,and JPEG, and

us ers may provide others, but tokeepexe cut ables smal l,decodersare not include d in an appli-

cation unlessexplicitlyrequeste d. The image.Decode func tioncon sults a table ofsup por ted

formats. Eachent ryinthe table speciﬁes fourthings: the nameofthe for mat; a str ing thatisa

preﬁxofall imagesencoded thisway,usedtodetec t theencoding; a function Decode that

de codes an enco dedimage;and another function DecodeConfig that deco des on lythe image

metadata,suchasits size andcolor space.Anent ryisadde d to the table bycal ling

image.RegisterFormat,typic ally fro m within the packageinitializer ofthe sup por tingpack-

agefor eachfor mat, like thisone in image/png:

package png // image/png

func Decode(r io.Reader) (image.Image, error)

func DecodeConfig(r io.Reader) (image.Config, error)

func init() {

const pngHeader = "\x89PNG\r\n\x1a\n"

image.RegisterFormat("png", pngHeader, Decode, DecodeConfig)

}

Theeffec t is thatanapp lic ationneed onlyblank-import the packagefor the for mat itneedsto

make the image.Decode func tionabletodecodeit.

The database/sql packageusesasimi lar mechanism tolet users ins tal l ju stthe dat abas e

dr ivers the y ne e d.For example:

import (

"database/mysql"

_"github.com/lib/pq" // enable support for Postgres

_"github.com/go-sql-driver/mysql" // enable support for MySQL

)

db, err = sql.Open("postgres", dbname) // OK

db, err = sql.Open("mysql", dbname) // OK

db, err = sql.Open("sqlite3", dbname) // returns error:

unknown driver "sqlite3"

Exercis e 10.1: Extend the jpeg prog ram so thatitconvertsany sup por ted inp utfor mat toany

output for mat, using image.Decode to detec t theinp utfor mat andaﬂag toselec t theout put

format.

Exercis e 10.2: Deﬁne a gener ic archive ﬁle-readingfunctioncap able ofreadingZIP ﬁles

(archive/zip)and POSIX tar ﬁles (archive/tar). Use a reg ist rat ionmechanism similarto

theone des crib edabove sothatsup por t foreach ﬁle for mat can beplugged inusingblank

imports.

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

SECTION 10.6. PACKAGES AND NAMING 289

10.6. Packages and Naming

In thissec tion,we’ll offer som e advice onhow tofol low Go’sdistinc tiveconvent ion s fornam-

ingpackages andtheir members.

Wh encre ating a package, keepits nameshort,but not soshort astobecrypt ic. Themost

frequentlyusedpackages in the stand ard librar y arenamed bufio, bytes, flag, fmt, http, io,

json, os, sort, sync,and time.

Be des crip tiveand unambiguous where possible. For example, don’tnameautilit y package

util when a namesuchas imageutil or ioutil is speciﬁcyet still con cis e.Avoid cho osing

packagenames thatare commonlyusedfor rel ate d lo cal variables, oryou may comp elthe

package’sclients touse renamingimp orts, as wit h the path package.

Packagenames usu allytakethe singu lar for m.The stand ard packages bytes, errors,and

strings us e theplural toavoid hidingthe cor respondingpre declare d typesand, in the cas e of

go/types,toavoid conﬂic t with a key word.

Av oid packagenames thatalready haveother connot ation s.For example, weoriginallyused

thename temp forthe temperatureconversionpackageinSec tion 2.5, but thatdidn’tlastlon g.

It was a ter r ibleide a becaus e ‘‘temp’’ is analmostunivers alsynonym for ‘‘temp orar y.’’ We went

thro ugh a brief per iodwit h thename temperature,but thatwas too lon g anddidn’tsay what

thepackagedid.Inthe end,itbecame tempconv,whichisshorter andparal lelwit h strconv.

No w let’stur n to the namingofpackagemembers.Since eachreference toamemberof

anot her packageusesaqualiﬁed identiﬁer suchas fmt.Println,the burden ofdes cribingthe

packagememberisbor neequ allybythe packagenameand the memb ername. Weneed not

ment ion the con ceptoffor matting in Println becaus e thepackagename fmt do es that

already.Whendesig ningapackage, con sider how the two par ts of a qualiﬁed identiﬁer wor k

together,not the memb ernamealone.Hereare som e ch arac ter ist icexamples:

bytes.Equal flag.Int http.Get json.Marshal

We c an identify som e common namingpatterns. The strings packageprovides a numberof

indep endentfunctions for manipu lat ingstr ings:

package strings

func Index(needle, haystack string) int

type Replacer struct{ /* ... */ }

func NewReplacer(oldnew ...string) *Replacer

type Reader struct{ /* ... */ }

func NewReader(s string) *Reader

Theword string do es notapp ear in anyoftheir names. Clients refer tothemas

strings.Index, strings.Replacer,and soon.

Ot her packages thatwemig htdes crib e as single-ty pepackages,suchas html/template and

math/rand,exp oseone princip aldat a type plu s itsmet hods, andoften a New func tiontocre-

ateins tances.

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

290 CHAPTER 10. PACKAGES AND THE GO TOOL

package rand // "math/rand"

type Rand struct{ /* ... */ }

func New(source Source) *Rand

Thiscan lead torep etit ion,asin template.Template or rand.Rand,whichiswhy the names

of these kinds ofpackages areoften especi ally short.

At the other extreme,there are packages like net/http that havealot ofnames wit houta lot

of str ucture, because the y perfor m acomplic ated task. Despit e having overtwent y typesand

many morefunctions,the package’smostimp ortantmembers havethe simplestnames: Get,

Post, Handle, Error, Client, Server.

10.7. The GoTool

Therestofthischapt ercon cer nsthe go to ol, whichisusedfor dow nlo ading, quer ying,

formatting , building, testing , andins tal lingpackages ofGocode.

The go to olcom binesthe featuresofadiverse setoftools int o on e commandset. Itisapack-

agemanager (analogou s to apt or rpm)thatanswers quer ies ab out itsinv ent ory ofpackages,

comp utestheir dependencies, anddow nlo adsthemfro m remote version-cont rol systems.Itis

abui ld systemthatcomputesﬁle dependencies andinv okescompi lers, assemb lers, andlin k-

ers, although itisint ent ion allylesscompletethanthe stand ard Unix make.And itisatest

dr iver, aswewill see inChapt er11.

Itscommand-lineint erface usesthe ‘‘Sw iss armyknife’’ st yle,wit h ov eradozen sub com-

mand s,som e of whichwehavealready seen, like get, run, build,and fmt.You can run

go help to see the index ofits bui lt-in document ation,but for reference,we’ve liste d themost

common lyusedcommand s below:

$go

...

build compile packages and dependencies

clean remove object files

doc show documentation for package or symbol

env print Go environment information

fmt run gofmt on package sources

get download and install packages and dependencies

install compile and install packages and dependencies

list list packages

run compile and run Go program

test test packages

version print Go version

vet run go tool vet on packages

Use "go help [command]" for more information about a command.

...

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

SECTION 10.7. THE GO TOOL 291

To keepthe need for conﬁgurat iontoa minimum, the go to olrelies heavi lyonconvent ion s.

Fo r example, given the nameofaGo source ﬁle,the toolcan ﬁnd its enclosingpackage,

becaus e each direc tor y cont ainsasinglepackageand the imp ort pat h of a packagecor-

resp ond s to the direc tor y hierarchyinthe wor ksp ace.Given the imp ort pat h of a package, the

to olcan ﬁnd the cor respondingdirec tor y in whichitstoresobj e ctﬁles. Itcan also ﬁnd the

URL ofthe ser ver thathosts the sourcecoderep ository.

10.7.1. WorkspaceOrganization

Theonlyconﬁgurat ionmostusers everneed isthe GOPATH enviro nment var iable,whichspeci-

ﬁes the rootofthe wor ksp ace.Whenswitchingtoadif ferentwor ksp ace,users updatethe

value of GOPATH.For ins tance,weset GOPATH to $HOME/gobook whilewor kingonthisbook:

$export GOPATH=$HOME/gobook

$goget gopl.io/...

Af ter you dow nlo ad al l theprogramsfor thisbookusingthe command above , your wor ksp ace

wi l l cont ain a hierarchylikethisone:

GOPATH/

src/

gopl.io/

.git/

ch1/

helloworld/

main.go

dup/

main.go

...

golang.org/x/net/

.git/

html/

parse.go

node.go

...

bin/

helloworld

dup

pkg/

darwin_amd64/

...

GOPATH hasthree sub direc tor ies. The src su bdirec tor y holdssourcecode. Eachpackage

resides in a direc tor y whos e name rel ative to $GOPATH/src is the package’simp ort pat h, such

as gopl.io/ch1/helloworld.Obs erve thatasingle GOPATH worksp ace cont ainsmultiplever-

sion-cont rol rep ositories beneath src,suchas gopl.io or golang.org.The pkg su bdirec tor y

is where the bui ld to ols store compi led packages, andthe bin su bdirec tor y holdsexe cut able

prog ramslike helloworld.

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

292 CHAPTER 10. PACKAGES AND THE GO TOOL

Asecon d enviro nment var iable, GOROOT,speciﬁes the rootdirec tor y of the Godistr ibution,

whichprovides allthe packages ofthe stand ard librar y.The direc tor y st ruc turebeneath

GOROOT resemb les thatof GOPATH,so, for example, the sourceﬁles ofthe fmt packagereside in

the $GOROOT/src/fmt direc tor y.Users never need toset GOROOT since,bydefau lt, the go to ol

wi l l us e thelocat ionwhere itwas inst alled.

The go env commandprintsthe effec tive values ofthe env iro nment var iables relevanttothe

to olchain, includingthe defau ltvalues for the missingones. GOOS sp eciﬁes the targetoperat-

ingsystem(forexample, android, linux, darwin,or windows)and GOARCH sp eciﬁes the target

processorarc hit ecture, suchas amd64, 386,or arm.Alt hough GOPATH is the onlyvar iable you

mu stset, the othersocc asionallyapp ear in our explanation s.

$goenv

GOPATH="/home/gopher/gobook"

GOROOT="/usr/local/go"

GOARCH="amd64"

GOOS="darwin"

...

10.7.2. Downloading Packages

Wh enusingthe go to ol, a package’simp ort pat h indic ates not onlywhere toﬁnd itinthe local

worksp ace,but where toﬁnd itonthe Int ernet sothat go get canret rie veand updateit.

The go get commandcan down loadasinglepackageoranent ire subtree orrep ository using

the ... notation, as in the pre vious sec tion.The toolals o comp utesand dow nlo adsall the

dep endencies ofthe initial packages, whichiswhy the golang.org/x/net/html package

appeared inthe wor ksp ace in the pre vious example.

Once go get hasdow nlo ade d thepackages, itbui ldsthemand then install s thelibrar ies and

commands.We’ll lookatthe det ails inthe next sec tion,but anexamplewill showhow

st raig htfor wardthe pro cessis. The ﬁrs t commandbelow gets the golint to ol, whichche cks

forcommonsty lepro blemsinGosourcecode. The secon d commandruns golint on

gopl.io/ch2/popcount from Sec tion 2.6.2. Ithelpf ullyrep ortsthatwehavefor g ott ento

wr ite a doc comment for the package:

$goget github.com/golang/lint/golint

$$GOPATH/bin/golint gopl.io/ch2/popcount

src/gopl.io/ch2/popcount/main.go:1:1:

package comment should be of the form "Package popcount ..."

The go get commandhas sup por t forpopu lar code-hosting sit eslikeGitHu b,Bitbucket, and

Launchpadand can makethe appro priaterequests totheir version-cont rol systems.For less

we ll-k now n sites, you may havetoindic atewhichversion-cont rol pro tocol touse inthe

import pat h, such asGit or Mercurial.Run go help importpath forthe det ails.

Thedirec tor ies that go get createsare trueclients ofthe remot e repository,not justcopies of

theﬁles, soyou can use version-cont rol command s to see a dif f of local edits you’ve made orto

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

SECTION 10.7. THE GO TOOL 293

up datetoadif ferentrevision. For example, the golang.org/x/net direc tor y is a Git client:

$cd$GOPATH/src/golang.org/x/net

$git remote -v

origin https://go.googlesource.com/net (fetch)

origin https://go.googlesource.com/net (push)

No tice thatthe app arent dom ain name in the package’simp ort pat h, golang.org,dif fersfro m

theactualdom ain nameofthe Git ser ver, go.googlesource.com.Thisisa feature ofthe go

to olthatlets packages use a customdom ain nameintheir import pat h whilebeinghoste d by a

genericser vice suchas googlesource.com or github.com.HTMLpages beneath

https://golang.org/x/net/html include the metadata shown below,whichredirec tsthe go

to oltothe Git rep ository atthe actualhosting sit e:

$gobuild gopl.io/ch1/fetch

$./fetch https://golang.org/x/net/html | grep go-import

content="golang.org/x/net git https://go.googlesource.com/net">

If you specif y the -u ﬂag , go get wi l l ensure thatall packages itvisits, includingdep endencies,

areupdated totheir latestversionbeforebeingbui ltand ins tal le d.Wit houtthatﬂag , packages

that already exist local lywill not beupdated.

The go get -u commandgeneral lyret rie ves the latestversionofeachpackage, whichiscon-

venientwhenyou’re getting started but may beinappro priatefor deploye d proj e cts, where

precis e cont rol ofdep endencies iscriticalfor release hyg iene. The usu alsolut ion tothis

problem isto vend or thecode, thatis, tomakeapersistentlocal copy ofall the necessary

dep endencies, andtoupdatethiscopycaref ullyand delib erately. Prior toGo1.5, thisrequired

ch ang ing those packages’ import pat hs, soour copy of golang.org/x/net/html would

become gopl.io/vendor/golang.org/x/net/html.Morerecentversions ofthe go to ol

supp ort vendoring direc tly,thoug h we don’thavespace toshowthe det ails here. See Vend or

Dire ctor ies in the out put of the go help gopath command.

Exercis e 10.3: Using fetch http://gopl.io/ch1/helloworld?go-get=1,ﬁnd out which

service hosts the codesamples for thisbook. (HTTP requests fro m go get include the go-get

parameter sothatser verscan dist inguish themfro m ordinar y browserrequests.)

10.7.3. Building Packages

The go build commandcompi les eachargumentpackage. Ifthe packageisalibrar y,the

resu ltisdis carde d; this merelyche cks thatthe packageisfre e of compi leerror s.Ifthe package

is name d main, go build invo kes the lin ker tocre ate anexe cut ableinthe cur rentdirec tor y;

thenameofthe exe cut ableistaken fro m thelastseg mentofthe package’simp ort pat h.

Sinceeachdirec tor y cont ainsone package, eachexe cut ableprogram, or comm and in Unix ter-

minolog y,requires its own direc tor y.These direc tor ies aresom etimeschi ldren ofadirec tor y

name d cmd,suchasthe golang.org/x/tools/cmd/godoc commandwhichser ves Gopack-

agedocumentation through a web int erface (§10.7.4).

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

294 CHAPTER 10. PACKAGES AND THE GO TOOL

Packages may bespeciﬁed bytheir import pat hs, as wesaw abov e,orbyarel ative direc tor y

name,whichmuststart wit h a . or .. segmentevenifthiswou ldnot ordinar ily berequired.

If noargumentisprovide d,the cur rentdirec tor y is assume d.Thu s thefol low ing command s

buildthe samepackage, thoug h each writesthe exe cut abletothe direc tor y in which go build

is run:

$cd$GOPATH/src/gopl.io/ch1/helloworld

$gobuild

and:

$cd

anywhere

$gobuild gopl.io/ch1/helloworld

and:

$cd$GOPATH

$gobuild ./src/gopl.io/ch1/helloworld

butnot:

$cd$GOPATH

$gobuild src/gopl.io/ch1/helloworld

Error: cannot find package "src/gopl.io/ch1/helloworld".

Packages may also bespeciﬁed asalistofﬁle names, thoug h this tends tobeusedonlyfor

smal l prog ramsand one-off exp eriments. Ifthe packagenameis main,the exe cut ablename

comesfro m thebas enameofthe ﬁrs t .go ﬁle.

$cat quoteargs.go

package main

import (

"fmt"

"os"

)

func main() {

fmt.Printf("%q\n", os.Args[1:])

}

$gobuild quoteargs.go

$./quoteargs one "two three" four\ five

["one" "two three" "four five"]

Partic ularlyfor throwaway programslikethisone,wewanttorun the exe cut ableassoonas

we’ve bui ltit. The go run commandcom binesthese two steps:

$gorun quoteargs.go one "two three" four\ five

["one" "two three" "four five"]

The ﬁrs t argumentthatdoesn’tend in .go is assume d to bethe beg inningofthe listofargu-

mentstothe Goexe cut able.

By defau lt, the go build commandbui ldsthe requeste d packageand all its dependencies, then

throws away allthe compi led codeexceptthe ﬁnalexe cut able, ifany.Bot h thedep endency

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

SECTION 10.7. THE GO TOOL 295

analysisand the compi lat ionare sur prisinglyfast, but aspro jec tsgrowtodozensofpackages

andhundredsofthous and s of lines ofcode, the timetorecompi ledep endencies can becom e

notice able,pot ent ial lyseveral secon ds, evenwhenthose dep endencies haven’tchange d at all.

The go install commandisver y simi lar to go build,exceptthatitsaves the compi led code

foreachpackageand command ins teadofthrow ing itaway.Compi led packages aresaved

bene ath the $GOPATH/pkg direc tor y correspondingtothe src direc tor y in whichthe source

resides, andcommand exe cut ables aresaved inthe $GOPATH/bin direc tor y.(Many users put

$GOPATH/bin on their exec utablesearc h path.) Thereafter, go build and go install do not

runthe compi ler for those packages andcommand s if the y have not change d,mak ingsubse-

quentbui ldsmuchfaster. For convenience, go build -i inst allsthe packages thatare dep en-

dencies ofthe bui ld target.

Sincecompi led packages varybyplatfor m andarc hit ecture, go install savesthembeneath a

su bdirec tor y whos e name incor porates the values ofthe GOOS and GOARCH enviro nment var i-

ables. For example, onaMac the golang.org/x/net/html packageiscompi led and ins tal le d

in the ﬁle golang.org/x/net/html.a under $GOPATH/pkg/darwin_amd64.

It isstraig htfor wardto cross-compi le aGoprogram, thatis, tobui ld an exec utableint ended for

adif ferentoperat ingsystemorCPU.Justset the GOOS or GOARCH var iables dur ingthe bui ld.

The cross prog ram pr intsthe operat ingsystemand arc hit ecturefor whichitwas bui lt:

gopl.io/ch10/cross

func main() {

fmt.Println(runtime.GOOS, runtime.GOARCH)

}

Thefol low ing command s produce 64-bit and 32-bit exe cut ables respectively:

$gobuild gopl.io/ch10/cross

$./cross

darwin amd64

$GOARCH=386 go build gopl.io/ch10/cross

$./cross

darwin 386

Some packages may need tocompi ledif ferentversions ofthe codefor cer tain platfor msor

processors,todealwit h low-le vel por tabilit y issues ortoprovide opt imize d versions of

importantroutines, for ins tance.Ifaﬁle nameincludes an operat ingsystemorpro cessor

archit ecturenamelike net_linux.go or asm_amd64.s,thenthe go to olwill compi lethe ﬁle

on lywhenbui ldingfor thattarget. Speci al comments cal le d buil d tags give moreﬁne-g rained

cont rol . Fo r example, ifaﬁle cont ainsthiscomment:

// +build linux darwin

before the packagedeclarat ion(andits doccomment), go build wi l l comp ile itonlywhen

buildingfor Linux orMac OS X, andthiscomment says never tocompi lethe ﬁle:

// +build ignore

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

296 CHAPTER 10. PACKAGES AND THE GO TOOL

Fo r more det ails,see the Bu ild Const rai nts sectionofthe go/build package’sdocumentation:

$godoc go/build

10.7.4. Documenting Packages

Go sty lestron gly encourages good documentation of packageAPIs. Eachdeclarat ionofan

exp orted packagememberand the packagedeclarat ionits elf shouldbeimmediate lypre ceded

by a comment explainingits pur pos e andusage .

Go doc comments arealways comp letesentences, andthe ﬁrs t sent enceisusu allyasummar y

that startswit h thenamebeingdeclare d.Functionparametersand other identiﬁersare men-

tion edwit houtquotation or marku p.For example, here’sthe doc comment for fmt.Fprintf:

// Fprintf formats according to a format specifier and writes to w.

// It returns the number of bytes written and any write error encountered.

func Fprintf(w io.Writer, format string, a ...interface{}) (int, error)

Thedet ails of Fprintf’sfor matting are explained inadoc comment ass oci ated wit h the fmt

packageits elf.Acommentimmediate lypre cedingapackage de clarat ioniscon sidered the

do c commentfor the packageasawhole.There mustbeonlyone,thoug h it may appear in

any ﬁle.Lon g erpackagecommentsmay warrantaﬁle oftheir own; fmt’sisover300 lines.

This ﬁle isusu allycal le d doc.go.

Go o d do cumentation need not beextensive , anddocumentation isnosubst itute for simplicity.

In deed,Go’sconvent ion s favor brevit y andsimplicity indocumentation as in all things, since

do cumentation,likecode, requires maintenancetoo.Manydeclarat ions can beexplained in

on e we ll-worde d sent ence, and ifthe beh avior istruly obv iou s,nocomment isneeded.

Throughoutthe book, as space per mits, we’ve pre ceded manydeclarat ions bydoc comments,

butyou will ﬁnd betterexamples as you brows e thestand ard librar y.Two tools can helpyou

do that.

The go doc to olprintsthe declarat ionand doc comment ofthe ent ity speciﬁed onthe com-

mand line, whichmay beapackage:

$godoc time

package time // import "time"

Package time provides functionality for measuring and displaying time.

const Nanosecond Duration = 1 ...

func After(d Duration) <-chan Time

func Sleep(d Duration)

func Since(t Time) Duration

func Now() Time

type Duration int64

type Time struct { ... }

...

many more

...

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

SECTION 10.7. THE GO TOOL 297

or a packagemember:

$godoc time.Since

func Since(t Time) Duration

Since returns the time elapsed since t.

It is shorthand for time.Now().Sub(t).

or a met hod:

$godoc time.Duration.Seconds

func (d Duration) Seconds() float64

Seconds returns the duration as a floating-point number of seconds.

Thetooldoesnot need completeimp ort pat hsorcor rec t identiﬁer cas e.Thiscommand prints

thedocumentation of (*json.Decoder).Decode from the encoding/json package:

$godoc json.decode

func (dec *Decoder) Decode(v interface{}) error

Decode reads the next JSON-encoded value from its input and stores

it in the value pointed to by v.

Thesecon d to ol, confusinglynamed godoc,ser ves cross-lin ked HTMLpages thatprovide the

same infor mat ionas go doc andmuchmore. The godoc server at https://golang.org/pkg

covers the stand ard librar y.Figure10.1 shows the documentation for the time package, and

in Sec tion 11.6 we’ll see godoc’sint erac tivedispl ayofexampleprograms. The godoc server at

https://godoc.org hasasearc hable index ofthous and s of open-s ource packages.

Yo u canals o runanins tance of godoc in yourwor ksp ace if you wanttobrows e your own

packages. Visit http://localhost:8000/pkg in yourbrows er whilerunningthiscommand:

$godoc -http :8000

Its -analysis=type and -analysis=pointer ﬂags augment the documentation and the

source code wit h theresults ofadvance d st aticanalysis.

10.7.5. Int ernal Packages

Thepackageisthe most imp ortantmechanism for enc apsulat ioninGoprograms. Unex-

ported identiﬁersare visible onlywit hin the samepackage, and exp orted identiﬁersare visible

to the wor ld.

Sometimes, thoug h, amidd le ground wou ldbehelpf ul, a way todeﬁne identiﬁersthatare visi-

bletoasmall set oftrusted packages, but not toeveryon e.For example, whenwe’re bre aking

up a large packageint o more manageablepar ts, we may not wanttorevealthe int erfaces

betweenthose par ts to other packages. Or we may wanttoshare utilit y func tions across

several packages ofapro jec t withoutexp osingthemmorewidely.Orperhaps wejustwantto

exp erimentwit h anew packagewit houtpremature lycommitt ing toits API, byput tingit ‘‘on

prob ation’’ with a limite d setofclients.

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

298 CHAPTER 10. PACKAGES AND THE GO TOOL

Figure 10.1. The time packagein godoc.

To addressthese needs, the go build to oltre ats a packagespeci ally ifits import pat h cont ains

apat h segmentnamed internal.Suchpackages arecal le d internal packages.Anint ernal

packagemay beimp orted onlybyanother packagethatisinside the tre e ro ote d at the parent

of the internal direc tor y.For example, given the packages below, net/http/inter-

nal/chunked canbeimp orted fro m net/http/httputil or net/http,but not fro m

net/url.How ever, net/url mayimp ort net/http/httputil.

net/http

net/http/internal/chunked

net/http/httputil

net/url

10.7.6. Querying Packages

The go list to olrep ortsinfor mat ionabout avai lable packages. Inits simplestfor m, go list

testswhether a packageispresent inthe wor ksp ace andprintsits import pat h if so:

$golist github.com/go-sql-driver/mysql

github.com/go-sql-driver/mysql

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

SECTION 10.7. THE GO TOOL 299

An argumentto go list maycontain the ‘‘...’’ wi ldc ard, whichmatch esany subst ringofa

package’simp ort pat h. We can use ittoenumerateall the packages wit hin a Gowor ksp ace:

$golist ...

archive/tar

archive/zip

bufio

bytes

cmd/addr2line

cmd/api

...

many more

...

or wit hin a speciﬁcsubtree:

$golist gopl.io/ch3/...

gopl.io/ch3/basename1

gopl.io/ch3/basename2

gopl.io/ch3/comma

gopl.io/ch3/mandelbrot

gopl.io/ch3/netflag

gopl.io/ch3/printints

gopl.io/ch3/surface

or rel ate d to a par tic ulartopic:

$golist ...xml...

encoding/xml

gopl.io/ch7/xmlselect

The go list commandobt ainsthe completemet adat a foreachpackage, not justthe imp ort

path,and makes thisinfor mat ionavai lable tousers orother tools inavar ietyoffor mats. The

-json ﬂag causes go list to print the ent ire record ofeachpackage in JSONfor mat:

$golist -json hash

{

"Dir": "/home/gopher/go/src/hash",

"ImportPath": "hash",

"Name": "hash",

"Doc": "Package hash provides interfaces for hash functions.",

"Target": "/home/gopher/go/pkg/darwin_amd64/hash.a",

"Goroot": true,

"Standard": true,

"Root": "/home/gopher/go",

"GoFiles": [

"hash.go"

"Imports": [

"io"

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

300 CHAPTER 10. PACKAGES AND THE GO TOOL

"Deps": [

"errors",

"io",

"runtime",

"sync",

"sync/atomic",

"unsafe"

]

}

The -f ﬂag lets users customize the out put for mat usingthe templ ate langu ageofpackage

text/template (§4.6). Thiscommand printsthe transitive dep endencies ofthe strconv

package, sep arated byspaces:

$golist -f '{{join .Deps " "}}' strconv

errors math runtime unicode/utf8 unsafe

andthiscommand printsthe direc t importsofeachpackageinthe compress su btree ofthe

st and ard librar y:

$golist -f '{{.ImportPath}} -> {{join .Imports " "}}' compress/...

compress/bzip2 -> bufio io sort

compress/flate -> bufio fmt io math sort strconv

compress/gzip -> bufio compress/flate errors fmt hash hash/crc32 io time

compress/lzw -> bufio errors fmt io

compress/zlib -> bufio compress/flate errors fmt hash hash/adler32 io

The go list commandisusefulfor bot h on e-off int erac tivequer ies andfor bui ld andtest

automation scr ipts. We’ll use itagain in Sec tion 11.2.4. For moreinfor mat ion, includingthe

setofavai lable ﬁelds and their meaning, see the out put of go help list.

In thischapt er, we’ve explained all the imp ortantsub command s of the go to ol—exceptone.In

thenext chapt er, we’ll see how the go test commandisusedfor testing Goprograms.

Exercis e 10.4: Cons truct a toolthatrep ortsthe set ofall packages in the wor ksp ace thattran-

sitive lydep endonthe packages speciﬁed bythe arguments. Hint: you will need torun

go list twice,oncefor the initial packages andoncefor all packages. You may wanttoparse

itsJSONout put usingthe encoding/json package(§4.5).

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

Te sting

Maur ice Wi l kes, the develop erofEDSAC, the ﬁrs t stored-program comp uter, had a startling

insig htwhi le climbing the stairsofhis lab orator y in 1949. In Memoir s of a ComputerPione er,

he recal le d, ‘‘Therealizat ioncameovermewit h full force thatagood par t of the remainder of

my lifewas going tobespent inﬁndingerror s in myown programs.’’ Sure lyevery program-

merofastored-program comp utersince thencan symp athize wit h Wi l kes, thoug h perh aps

notwit houtsom e bemu sement at his naïvetéabout the difﬁcult ies ofsof twarecon str uction.

Prog ramstoday arefar largerand more complex thaninWilkes’stime, ofcours e, andagre at

de al of effor t hasbeenspent ontechniques tomakethiscomplexity manageable. Two

te chniques in par tic ularstand out for their effe ctiveness.The ﬁrs t is routine peerrevie w of

prog ramsbeforethe y aredeploye d.The secon d,the subjec t of thischapt er, istesting .

Test ing , by whichweimp licitlymean au tom ate d test ing , is the prac tice ofwriting small

prog ramsthatche ckthatthe codeunder test(the produc tion co de) behavesasexp ected for

cer tain inputs, whichare usu allyeit her caref ullychosentoexercis e cer tain featuresorran-

domize d to ens ure bro adcov erage.

The ﬁeldofsof twaretesting isenormou s.The taskoftesting occ upies allprogrammerssom e

of the timeand som e prog rammersall ofthe time. The lit eratureontesting includes

thou sands ofprint edbooks andmillions ofwords ofblog posts. Inevery mainst ream

prog ramminglangu age, there are dozensofsof twarepackages intended for testcon str uction,

some wit h agre atdealofthe ory,and the ﬁeldseems toatt rac t more thanafew pro phets wit h

cult-li kefol low ings. Itisalmostenoug h to convinceprogrammersthattowrite effec tive tests

they mustacquireawhole new setofski l ls.

Go’sappro ach to test ing can seemrat her low-techincomparison. Itrelies onone command,

go test,and a set ofconvent ion s forwriting testfunctions that go test canrun. Thecom-

parat ive lylig htweig htmechanism iseffec tive for puretesting , anditextends natural lyto

benchmarks andsystematicexamples for documentation.

301

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

302 CHAPTER 11. TESTING

In prac tice,writing testcodeisnot muchdif ferentfro m wr iting the originalprogram its elf.

We w rite short functions thatfocus onone par t of the task. Wehavetobecaref ulofbound ary

condit ion s,thin k ab out dat a st ruc tures, andreasonabout whatresults a comp utation should

produce fro m suit ableinp uts. But thisisthe samepro cessaswriting ordinar y Go code; it

ne e dn’trequirenew not ation s,convent ion s,and tools.

11.1. The go test To ol

The go test su bcommandisatestdriverfor Gopackages thatare organize d accordingtocer-

tain conv ent ion s.Inapackagedirec tor y,ﬁles whose names endwit h _test.go arenot par t of

thepackageordinar ily bui ltby go build butare a par t of it whenbui ltby go test.

Wi thin *_test.go ﬁles, three kinds offunctions are tre ate d sp eci ally : tests, benchmarks, and

examples. A te st func tion,whichisafunctionwhose namebeg inswit h Test,exercis essom e

prog ram log ic forcor rec t behavior ; go test callsthe testfunctionand rep ortsthe result,

whichiseit her PASS or FAIL.Abenchm ark fun cti on hasanamebeg inningwit h Benchmark

andmeasuresthe per for mance ofsom e op erat ion; go test reportsthe mean exec ution time

of the operat ion. And an ex amp lefun cti on,whose namestartswit h Example,provides

machine-che cke d do cumentation.Wewill cov ertests in detai l in Sec tion 11.2, benchmarksin

Section11.4, andexamples in Sec tion 11.6.

The go test to olscans the *_test.go ﬁles for these speci al func tions,generates a temporar y

main packagethatcal lsthemall inthe pro per way,bui ldsand runsit, rep ortsthe results, and

then cle ansup.

11.2. Test Functions

Each test ﬁle mustimp ort the testing package. Testfunctions havethe fol low ing sig nature:

func Test

Name

(t *testing.T) {

// ...

}

Test functionnames mustbeg in with Test;the opt ion alsufﬁx

Name

mu stbeg in with a capit al

letter:

func TestSin(t *testing.T) { /* ... */ }

func TestCos(t *testing.T) { /* ... */ }

func TestLog(t *testing.T) { /* ... */ }

The t parameter provides met hodsfor rep ortingtestfai lures andlog gingaddition al

infor mat ion. Let’sdeﬁne anexamplepackage gopl.io/ch11/word1,containinga singlefunc-

tion IsPalindrome that rep ortswhether a str ing reads the samefor wardand backward. (This

implementation tests every bytetwice if the str ing isapalindrom e;we’ll com e back to that

shortly.)

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

SECTION 11.2. TEST FUNCTIONS 303

gopl.io/ch11/word1

// Package word provides utilities for word games.

package word

// IsPalindrome reports whether s reads the same forward and backward.

// (Our first attempt.)

func IsPalindrome(s string) bool {

for i := range s {

if s[i] != s[len(s)-1-i] {

return false

}

return true

}

In the samedirec tor y,the ﬁle word_test.go cont ainstwo testfunctions named TestPalin-

drome and TestNonPalindrome.Eachche cks that IsPalindrome givesthe rig htanswerfor a

singleinp utand rep ortsfai lures using t.Error:

package word

import "testing"

func TestPalindrome(t *testing.T) {

if !IsPalindrome("detartrated") {

t.Error(`IsPalindrome("detartrated") = false`)

}

if !IsPalindrome("kayak") {

t.Error(`IsPalindrome("kayak") = false`)

}

func TestNonPalindrome(t *testing.T) {

if IsPalindrome("palindrome") {

t.Error(`IsPalindrome("palindrome") = true`)

}

A go test (or go build)command wit h no packagearguments operates onthe packagein

thecur rentdirec tor y.Wecan bui ld andrun the tests wit h thefol low ing command.

$cd$GOPATH/src/gopl.io/ch11/word1

$gotest

ok gopl.io/ch11/word1 0.008s

Sat isﬁed,weshipthe program, but nosoonerhavethe launch par tyguests depar ted thanthe

bugrep ortsstart toarrive . AFrenchusernamed NoelleEve Elleoncompl ainsthat IsPalin-

drome do esn’trecog nize ‘‘été.’’ Anot her,fro m Cent ral America,isdis app ointe d that itrej e cts

‘‘Aman, a plan, a canal: Panama.’’ Thes e sp eciﬁcand small bug rep ortsnatural lylendthem-

selves tonew testcas es.

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

304 CHAPTER 11. TESTING

func TestFrenchPalindrome(t *testing.T) {

if !IsPalindrome("été") {

t.Error(`IsPalindrome("été") = false`)

}

func TestCanalPalindrome(t *testing.T) {

input := "A man, a plan, a canal: Panama"

if !IsPalindrome(input) {

t.Errorf(`IsPalindrome(%q) = false`,input)

}

To avoid writing the lon g input st ringtwice,weuse Errorf,whichprovides for matting like

Printf.

Wh enthe two new tests havebeenadde d,the go test commandfai lswit h infor mat ive error

mess ages.

$gotest

--- FAIL: TestFrenchPalindrome (0.00s)

word_test.go:28: IsPalindrome("été") = false

--- FAIL: TestCanalPalindrome (0.00s)

word_test.go:35: IsPalindrome("A man, a plan, a canal: Panama") = false

FAIL

FAIL gopl.io/ch11/word1 0.014s

It’sgood prac tice towrite the test ﬁrs t andobs erve thatittriggersthe samefai luredes crib edby

theuser’sbug rep ort.Onlythencan webeconﬁdentthatwhate ver ﬁxwecom e up wit h

addressesthe rig htpro blem.

As a bonus,running go test is usuallyquickerthanmanuallygoing through the steps

des crib edinthe bug rep ort,allow ing ustoiteratemorerapid ly. Ifthe testsuite containsmany

slow tests, wemay makeevenfasterprogress if we’re selec tive about whichoneswerun.

The -v ﬂag printsthe nameand exe cut ion timeofeachtest in the package:

$gotest -v

=== RUN TestPalindrome

--- PASS: TestPalindrome (0.00s)

=== RUN TestNonPalindrome

--- PASS: TestNonPalindrome (0.00s)

=== RUN TestFrenchPalindrome

--- FAIL: TestFrenchPalindrome (0.00s)

word_test.go:28: IsPalindrome("été") = false

=== RUN TestCanalPalindrome

--- FAIL: TestCanalPalindrome (0.00s)

word_test.go:35: IsPalindrome("A man, a plan, a canal: Panama") = false

FAIL

exit status 1

FAIL gopl.io/ch11/word1 0.017s

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

SECTION 11.2. TEST FUNCTIONS 305

andthe -run ﬂag , whos e argumentisaregu lar expression, causes go test to run onlythose

testswhose functionnamematch esthe pattern:

$gotest -v -run="French|Canal"

=== RUN TestFrenchPalindrome

--- FAIL: TestFrenchPalindrome (0.00s)

word_test.go:28: IsPalindrome("été") = false

=== RUN TestCanalPalindrome

--- FAIL: TestCanalPalindrome (0.00s)

word_test.go:35: IsPalindrome("A man, a plan, a canal: Panama") = false

FAIL

exit status 1

FAIL gopl.io/ch11/word1 0.014s

Of course,oncewe’ve gott enthe selec ted tests topass, weshouldinv oke go test with no ﬂags

to run the ent ire testsuite one lasttimebeforewecommitthe change .

No w ourtaskistoﬁxthe bugs. A quickinv est igat ionreveals the cause ofthe ﬁrs t bugtobe

IsPalindrome’suse ofbytesequences, not runesequences, sothatnon-ASCII charac terssuch

as the é in "été" conf use it. Thesecon d bugarisesfro m notignor ing spaces, punctuation,

andlettercas e.

Chastene d,werewrite the functionmorecaref ully:

gopl.io/ch11/word2

// Package word provides utilities for word games.

package word

import "unicode"

// IsPalindrome reports whether s reads the same forward and backward.

// Letter case is ignored, as are non-letters.

func IsPalindrome(s string) bool {

var letters []rune

for _, r := range s {

if unicode.IsLetter(r) {

letters = append(letters, unicode.ToLower(r))

}

for i := range letters {

if letters[i] != letters[len(letters)-1-i] {

return false

}

return true

}

We als o wr ite a morecomprehensiveset oftestcas es that combines allthe pre vious onesand a

numb erofnew onesint o atable.

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

306 CHAPTER 11. TESTING

func TestIsPalindrome(t *testing.T) {

var tests = []struct {

input string

want bool

}{

{"", true},

{"a", true},

{"aa", true},

{"ab", false},

{"kayak", true},

{"detartrated", true},

{"A man, a plan, a canal: Panama", true},

{"Evil I did dwell; lewd did I live.", true},

{"Able was I ere I saw Elba", true},

{"été", true},

{"Et se resservir, ivresse reste.", true},

{"palindrome", false}, // non-palindrome

{"desserts", false}, // semi-palindrome

}

for _, test := range tests {

if got := IsPalindrome(test.input); got != test.want {

t.Errorf("IsPalindrome(%q) = %v", test.input, got)

}

Ournew tests pass:

$gotest gopl.io/ch11/word2

ok gopl.io/ch11/word2 0.015s

Thissty leof tabl e-driven test ing isver y common inGo. Itisstraig htfor wardtoadd new table

entr ies as needed,and since the ass ertionlog ic is not dup lic ated,wecan invest moreeffor t in

producingagood error message .

Theout put of a fai lingtestdoes not include the ent ire stack trace at the moment ofthe cal l to

t.Errorf.Nor does t.Errorf caus e apanic orstopthe exe cut ion of the test, unli keass ertion

fai lures in manytestframewor ksfor other langu ages. Tests areindep endentofeachother.If

an early ent ryinthe table causesthe testtofai l,later table entr ies will still beche cke d,and

thus wemay lear n ab out multiplefai lures dur ingasinglerun.

Wh enwereallymuststopatestfunction, perhaps because some initializat ioncodefai le d or to

preventafai lurealready rep orted fro m causingaconfusingcas cade ofothers, weuse t.Fatal

or t.Fatalf.These mustbecal le d from the samegoroutine asthe Test func tion, not fro m

anot her one cre ate d during the test.

Test fai luremessagesare usu allyofthe for m "f(x) = y, want z",where f(x) explainsthe

attemp ted operat ionand its input, y is the actualresult, and z theexp ected result. Where con-

venient, as in our palindrom e example, actualGosyntaxisusedfor the f(x) part.Displ aying

x is par tic ularlyimp ortant in a table-dr iventest, since a given assertionisexe cut edmany

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

SECTION 11.2. TEST FUNCTIONS 307

timeswit h dif ferentvalues. Avoid boi ler plate and redundant infor mat ion. Whentesting a

boole an func tionsuchas IsPalindrome,omitthe want z part since itaddsnoinfor mat ion. If

x, y,or z is lengt hy, print a con cis e summar y of the relevantpar ts instead. The aut hor of a test

shouldstr ive tohelpthe programmer who mustdiagnos e atestfai lure.

Exercis e 11.1: Wr ite testsfor the charcount prog ram in Sec tion 4.3.

Exercis e 11.2: Wr ite a set oftests for IntSet (§6.5) thatche cks thatits beh avior after each

op erat ionisequivalenttoaset bas edonbui lt-in maps. Saveyourimp lementation for

benchmarking inExercis e 11.7.

11.2.1. Randomized Testing

Table-dr iventests areconvenientfor che cking thatafunctionwor ksoninp uts caref ully

sele cte d to exercis e interest ing cas es in the log ic. Anot her approach, ra ndomizedtesting,

exploresabro ader range ofinp uts bycon str uctinginp uts at random.

Ho w do weknowwhatout put toexp ect fro m ourfunction, given a randominp ut? Thereare

twostrateg ies. The ﬁrs t is towrite analt ernat ive imp lementation of the functionthatusesa

lessefﬁcientbut simpler andcle arer algor it hm, andche ckthatbot h implementation s give the

same result. Thesecon d is tocre ate inp utvalues accordingtoapattern sothatweknowwhat

output toexp ect.

Theexamplebelow usesthe secon d approach:the randomPalindrome func tiongenerates

word s that areknown tobepalindrom esbycon str uction.

import "math/rand"

// randomPalindrome returns a palindrome whose length and contents

// are derived from the pseudo-random number generator rng.

func randomPalindrome(rng *rand.Rand) string {

n:=rng.Intn(25) // random length up to 24

runes := make([]rune, n)

for i := 0; i < (n+1)/2; i++ {

r:=rune(rng.Intn(0x1000)) // random rune up to '\u0999'

runes[i] = r

runes[n-1-i] = r

}

return string(runes)

}

func TestRandomPalindromes(t *testing.T) {

// Initialize a pseudo-random number generator.

seed := time.Now().UTC().UnixNano()

t.Logf("Random seed: %d", seed)

rng := rand.New(rand.NewSource(seed))

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

308 CHAPTER 11. TESTING

for i := 0; i < 1000; i++ {

p:=randomPalindrome(rng)

if !IsPalindrome(p) {

t.Errorf("IsPalindrome(%q) = false", p)

}

Sincerandomize d testsare non deter minist ic, itiscriticalthatthe log ofthe fai lingtestrecord

sufﬁcientinfor mat iontorepro duce the fai lure. Inour example, the inp ut p to IsPalindrome

te lls us all weneed toknow, but for functions thatacceptmorecomplex inputs, itmay besim-

pler tolog the seed ofthe pseudo-randomnumbergenerator (as wedoabove)thantodump

theent ire inp utdat a st ruc ture. Armed wit h that seed value,wecan easi lymodif y thetestto

repl aythe fai luredeter minist ically.

By usingthe cur renttimeasasourceofrandomness, the testwill explore nov elinp uts each

time itisrun, overthe ent ire cours e of itslifet ime. Thisisesp eci ally valuableifyourpro jec t

us es an aut omated systemtorun allits tests per iodic ally.

Exercis e 11.3: TestRandomPalindromes on lytests palindrom es. Write a randomize d test that

generates andver iﬁes non-palindrom es.

Exercis e 11.4: Mo dif y randomPalindrome to exercis e IsPalindrome’shandlingofpunc-

tu ation and spaces.

11.2.2. Testing a Command

The go test to olisusefulfor testing librar y packages, but wit h alit tle effort wecan use itto

test command s as wel l.Apackagenamed main ordinar ily pro duces an exec utableprogram,

butitcan beimp orted asalibrar y to o.

Let’swrite a test for the echo prog ram of Sec tion 2.3.2. We’ve split the program into two func-

tion s: echo do es therealwor k,whi le main pars esand reads the ﬂag values andrep ortsany

er ror s returned by echo.

gopl.io/ch11/echo

// Echo prints its command-line arguments.

package main

import (

"flag"

"fmt"

"io"

"os"

"strings"

)

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

SECTION 11.2. TEST FUNCTIONS 309

var (

n=flag.Bool("n", false, "omit trailing newline")

s=flag.String("s", " ", "separator")

)

var out io.Writer = os.Stdout // modified during testing

func main() {

flag.Parse()

if err := echo(!*n, *s, flag.Args()); err != nil {

fmt.Fprintf(os.Stderr, "echo: %v\n", err)

os.Exit(1)

}

func echo(newline bool, sep string, args []string) error {

fmt.Fprint(out, strings.Join(args, sep))

if newline {

fmt.Fprintln(out)

}

return nil

}

Fr omthe test, wewill cal l echo with a var ietyofarguments and ﬂag settingsand che ckthatit

pr intsthe cor rec t output ineachcas e,sowe’ve adde d parametersto echo to reduce its depen-

dence onglobalvar iables. That said,we’ve als o introduce d anot her globalvar iable, out,the

io.Writer to whichthe resultwill bewritt en. Byhav ing echo wr ite through thisvar iable,not

direc tly to os.Stdout,the tests can subst itute a dif ferent Writer implementation thatrecords

what was writt enfor later insp ection. Here’sthe test, in ﬁle echo_test.go:

package main

import (

"bytes"

"fmt"

"testing"

)

func TestEcho(t *testing.T) {

var tests = []struct {

newline bool

sep string

args []string

want string

}{

{true, "", []string{}, "\n"},

{false, "", []string{}, ""},

{true, "\t", []string{"one", "two", "three"}, "one\ttwo\tthree\n"},

{true, ",", []string{"a", "b", "c"}, "a,b,c\n"},

{false, ":", []string{"1", "2", "3"}, "1:2:3"},

}

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

310 CHAPTER 11. TESTING

for _, test := range tests {

descr := fmt.Sprintf("echo(%v, %q, %q)",

test.newline, test.sep, test.args)

out = new(bytes.Buffer) // captured output

if err := echo(test.newline, test.sep, test.args); err != nil {

t.Errorf("%s failed: %v", descr, err)

continue

}

got := out.(*bytes.Buffer).String()

if got != test.want {

t.Errorf("%s = %q, want %q", descr, got, test.want)

}

No tice thatthe testcodeisinthe samepackageasthe pro duc tioncode. Alt hough the package

name is main anditdeﬁnesamain func tion, dur ingtesting thispackageactsasalibrar y that

exp oses the function TestEcho to the testdriver; its main func tionisignored.

By organizingthe testasatable,wecan easi lyadd new testcas es. Let’ssee whathappens when

thetestfai ls, byaddingthislinetothe table:

{true, ",", []string{"a", "b", "c"}, "a b c\n"}, // NOTE: wrong expectation!

go test pr ints

$gotest gopl.io/ch11/echo

--- FAIL: TestEcho (0.00s)

echo_test.go:31: echo(true, ",", ["a" "b" "c"]) = "a,b,c", want "a b c\n"

FAIL

FAIL gopl.io/ch11/echo 0.006s

Theerror message des crib esthe att emp ted operat ion(usingGo-li kesyntax), the actualbeh av-

ior, and the exp ected beh avior,inthatorder.Wit h an infor mat ive error message suchasthis,

youmay haveaprett y go o d ide a ab out the rootcause beforeyou’ve evenlocated the source

co de of the test.

It’simp ortantthatcodebeingteste d notcal l log.Fatal or os.Exit,since these will stopthe

processinits tracks; cal lingthese functions shouldberegarde d as the exc lusiverig htof main.

If som ethingtot allyunexp ected happens and a functionpanics, the testdriverwill recov er,

though the testwill ofcours e be con sidered a fai lure. Exp ected error s such asthose resulting

from bad userinp ut, missing ﬁles, orimp rop erconﬁgurat ionshouldberep orted byretur ning

anon-ni l error value.For tunately(though unfor tunateasanillustrat ion), our echo example

is sosimplethatitwill never retur n anon-ni l er ror.

11.2.3. White-Box Testing

Oneway ofcategor izingtests isbythe level ofknowledge the y re quireofthe int ernal wor kings

of the packageunder test. A bl ack-box test assumesnot hingabout the packageother than

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

SECTION 11.2. TEST FUNCTIONS 311

what isexp osed byits APIand speciﬁed byits document ation;the package’sint ernalsare

op aque. Incontrast, a whit e-box test has privi leged accesstothe int ernal functions and dat a

st ruc tures ofthe packageand can makeobs ervat ions and changesthatanordinar y clientcan-

not. For example, a white-b oxtestcan che ckthatthe invar iants ofthe package’sdat a typesare

maintained after every operat ion. (Thename whit e box is tradition al, but cl ear box wouldbe

more acc urate.)

Thetwo appro ach esare complementary.Black-b oxtests areusu allymorerobust, needing

fe wer updates as the sof twareevo l ves. They als o helpthe testaut hor emp athize wit h theclient

of the packageand can reveal ﬂaws in the API desig n. In contrast, white-b oxtests can provide

more det aile d coverageofthe trick ier par ts of the imp lementation.

We’ve already seenexamples ofbot h kind s. TestIsPalindrome callsonlythe exp orted func-

tion IsPalindrome andisthu s ablack-b oxtest. TestEcho callsthe echo func tionand

up dates the globalvar iable out,bot h of whichare unexp orted,mak ingitawhite-b oxtest.

Whiledeveloping TestEcho,wemodiﬁed the echo func tiontouse the package-le vel var iable

out when writing its out put,sothatthe testcou ldreplace the stand ard out put wit h an alter-

native imp lementation thatrecords the dat a forlater insp ection. Usingthe sametechnique,

we can replace other par ts of the pro duc tioncodewit h easy-to-test ‘‘fake’’ implementation s.

Theadvantage offakeimp lementation s is thatthe y canbesimpler toconﬁgure, more

predic table,morereliable, and easier toobs erve . They can also avoid undesirable side effe cts

such asupdat ingapro duc tiondat abas e or charg ing a cre dit card.

Thecodebelow shows the quota-che cking log ic in a web ser vice thatprovides net wor ked

storagetousers.Whenusers exceed 90% oftheir quota,the systemsends themawar ning

emai l.

gopl.io/ch11/storage1

package storage

import (

"fmt"

"log"

"net/smtp"

)

func bytesInUse(username string) int64 { return 0 /* ... */ }

// Email sender configuration.

// NOTE: never put passwords in source code!

const sender = "notifications@example.com"

const password = "correcthorsebatterystaple"

const hostname = "smtp.example.com"

const template = `Warning: you are using %d bytes of storage,

%d%% of your quota.`

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

312 CHAPTER 11. TESTING

func CheckQuota(username string) {

used := bytesInUse(username)

const quota = 1000000000 // 1GB

percent := 100 * used / quota

if percent < 90 {

return // OK

}

msg := fmt.Sprintf(template, used, percent)

auth := smtp.PlainAuth("", sender, password, hostname)

err := smtp.SendMail(hostname+":587", auth, sender,

[]string{username}, []byte(msg))

if err != nil {

log.Printf("smtp.SendMail(%s) failed: %s", username, err)

}

We’dliketotestit, but wedon’twantthe testtosendout realemail.Sowemov e theemail

log ic into its own functionand store thatfunctioninanunexp orted package-le vel var iable,

notifyUser.

gopl.io/ch11/storage2

var notifyUser = func(username, msg string) {

auth := smtp.PlainAuth("", sender, password, hostname)

err := smtp.SendMail(hostname+":587", auth, sender,

[]string{username}, []byte(msg))

if err != nil {

log.Printf("smtp.SendEmail(%s) failed: %s", username, err)

}

func CheckQuota(username string) {

used := bytesInUse(username)

const quota = 1000000000 // 1GB

percent := 100 * used / quota

if percent < 90 {

return // OK

}

msg := fmt.Sprintf(template, used, percent)

notifyUser(username, msg)

}

We can now write a test thatsubst itutesasimplefakenot iﬁcation mechanism insteadofsend-

ingrealemail.Thisone records the notiﬁed userand the contentsofthe mess age .

package storage

import (

"strings"

"testing"

)

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

SECTION 11.2. TEST FUNCTIONS 313

func TestCheckQuotaNotifiesUser(t *testing.T) {

var notifiedUser, notifiedMsg string

notifyUser = func(user, msg string) {

notifiedUser, notifiedMsg = user, msg

}

// ...simulate a 980MB-used condition...

const user = "joe@example.org"

CheckQuota(user)

if notifiedUser == "" && notifiedMsg == "" {

t.Fatalf("notifyUser not called")

}

if notifiedUser != user {

t.Errorf("wrong user (%s) notified, want %s",

notifiedUser, user)

}

const wantSubstring = "98% of your quota"

if !strings.Contains(notifiedMsg, wantSubstring) {

t.Errorf("unexpected notification message <<%s>>, "+

"want substring %q", notifiedMsg, wantSubstring)

}

There’sone pro blem: afterthistestfunctionhas retur ned, CheckQuota no lon g erwor ksasit

shouldbecause it’sstill usingthe test’sfakeimp lementation of notifyUsers.(Thereisalways

ariskofthiskindwhenupdat ingglobalvar iables.) Wemustmodif y thetesttorestore the

previous value sothatsubsequenttests obs erve noeffec t,and wemustdothisonall exe cut ion

paths, includingtestfai lures andpanics. Thisnatural lysug gests defer.

func TestCheckQuotaNotifiesUser(t *testing.T) {

// Save and restore original notifyUser.

saved := notifyUser

defer func() { notifyUser = saved }()

// Install the test'sfake notifyUser.

var notifiedUser, notifiedMsg string

notifyUser = func(user, msg string) {

notifiedUser, notifiedMsg = user, msg

}

// ...rest of test...

}

Thispattern can beusedtotemporar ily saveand restore all kinds ofglobalvar iables, including

command-line ﬂags, debuggingopt ion s,and per for mance parameters; toins tal l andremov e

ho oks thatcause the pro duc tioncodetocal l some testcodewhensom ethingint erest ing hap-

pens;and tocoaxthe pro duc tioncodeint o rarebut imp ortantstates, such as timeouts, erro rs,

andevenspeciﬁcint erleavingsofcon cur rentactiv ities.

Usingglobalvar iables in thisway issafeonlybecause go test do es notnor mal lyrun multiple

testscon cur rently.

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

314 CHAPTER 11. TESTING

11.2.4. External Test Packages

Consider the packages net/url,whichprovides a URL parser, and net/http,whichprovides

aweb ser ver andHTTPclientlibrar y.Aswemig htexp ect,the hig her-le vel net/http dep ends

on the low er-le vel net/url.How ever, one ofthe tests in net/url is anexampledemon strat-

ingthe int erac tionbet weenURLs andthe HTTPclientlibrar y.Inother words,atestofthe

lower-le vel packageimp ortsthe hig her-le vel package.

Figure 11.1. Atestof net/url dep ends on net/http.

Decl aring thistestfunctioninthe net/url packagewou ldcre ate a cyc leinthe package

import graph, as depic ted bythe upwards arrow inFigure11.1, but asweexplained in

Section10.1, the Gospeciﬁcation forbids imp ort cyc les.

We res olvethe pro blem bydeclaring the testfunctioninan exter nal testpackage,thatis, in a

ﬁle in the net/url direc tor y whos e packagedeclarat ionreads package url_test.The ext ra

sufﬁx _test is a sig nal to go test that itshouldbui ld an addition alpackagecontainingjust

thes e ﬁles andrun its tests. Itmay behelpf ultothin k of thisexter nal testpackageasifithad

theimp ort pat h net/url_test,but it cannot beimp orted under thisorany other name.

Becaus e exter nal tests live inasep aratepackage, the y mayimp ort helperpackages thatals o

dep endonthe packagebeingteste d;anin-p ackagetestcannot dothis. Inter msofthe desig n

layers,the exter nal testpackageislog ical lyhig her upthanbot h of the packages itdep ends

up on, as shown inFigure11.2.

Figure 11.2. External testpackages bre akdep endency cycles.

By avoidingimp ort cyc les, exter nal testpackages allowtests, especi ally inte grati ontests (w hich

test the int erac tionofseveral comp onents), toimp ort other packages fre ely,exac tly asan

applic ationwou ld.

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

SECTION 11.2. TEST FUNCTIONS 315

We c an use the go list to oltosummar ize whichGosourceﬁles in a packagedirec tor y are

produc tioncode, in-p ackagetests, andexter nal tests. We’ll use the fmt packageasanexample.

GoFiles is the listofﬁles thatcontain the pro duc tioncode; these are the ﬁles that go build

wi l l include in yourapp lic ation:

$golist -f={{.GoFiles}} fmt

[doc.go format.go print.go scan.go]

TestGoFiles is the listofﬁles thatals o belong tothe fmt package, but these ﬁles, whose

namesall end in _test.go,are include d on lywhenbui ldingtests:

$golist -f={{.TestGoFiles}} fmt

[export_test.go]

Thepackage’stests wou ldusu allyreside in these ﬁles, thoug h unusually fmt hasnon e;we’ll

explain the pur pos e of export_test.go in a mom ent.

XTestGoFiles is the listofﬁles thatcon stitute the exter nal testpackage, fmt_test,sothese

ﬁles mustimp ort the fmt packageinorder touse it. Again, the y areinclude d on lydur ingtest-

ing:

$golist -f={{.XTestGoFiles}} fmt

[fmt_test.go scan_test.go stringer_test.go]

Sometimesanexter nal testpackagemay need privi leged accesstothe int ernalsofthe package

under test, if for exampleawhite-b oxtestmustliveinasep aratepackagetoavoid an import

cycle. Insuchcas es, we use a trick:weadd decl arat ions toanin-p ackage _test.go ﬁle to

exp osethe necessary int ernalstothe exter nal test. This ﬁle thu s of fersthe testa‘‘back door’’

to the package. Ifthe sourceﬁle existsonlyfor thispur pos e andcontainsnotests its elf,itis

of ten cal le d export_test.go.

Fo r example, the imp lementation of the fmt packageneedsthe functionality of unicode.Is-

Space as par t of fmt.Scanf.Toavoid creating anundesirable dependency, fmt do es not

import the unicode packageand its large tables ofdat a; instead, itcontainsasimpler imple-

ment ation,whichitcal ls isSpace.

To ens ure thatthe beh avior s of fmt.isSpace and unicode.IsSpace do not drift apart, fmt

pr udentlycontainsatest. Itisanexter nal test, andthu s it cannot access isSpace direc tly,so

fmt op ens a backdoortoitbydeclaring anexp orted var iable thatholds the int ernal isSpace

func tion. Thisisthe ent irety ofthe fmt package’s export_test.go ﬁle.

package fmt

var IsSpace = isSpace

Thistest ﬁle deﬁnesnotests; itjustdeclaresthe exp orted symbol fmt.IsSpace foruse bythe

exter nal test. Thistrick can also beusedwhene ver an exter nal testneedstouse some ofthe

te chniques ofwhite-b oxtesting .

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

316 CHAPTER 11. TESTING

11.2.5. Writing EffectiveTests

Many newcomers to Goare sur prisedbythe minimalism ofGo’stesting framewor k.Other

languages’ framewor ksprovide mechanismsfor identifyingtestfunctions (of ten usingreﬂec-

tion or met adat a), ho oks for per for ming ‘‘setup’’ and ‘‘te ardow n’’ op erat ions beforeand after

thetests run, andlibrar ies of utilit y func tions for ass ertingcommonpre dic ates, comp aring

values, for matting error messages, andabortingafai le d test (of ten usingexception s).

Although these mechanismscan maketests ver y concis e,the resulting tests often seemlike

they are writt eninaforeig n language . Furthermore, alt hough the y mayrep ort PASS or FAIL

correc tly,their manner may beunf riendlytothe unfor tunatemaintainer,wit h cr ypticfai lure

mess ageslike "assert: 0 == 1" or pageafter pageofstack traces.

Go’satt itude totesting stand s in stark contrast. Itexp ectstestaut hor s to domostofthiswor k

themselves, deﬁningfunctions toavoid rep etit ion,justasthe y wouldfor ordinar y prog rams.

Thepro cessoftesting isnot one ofrot e form ﬁlling; a testhas a userint erface too,alb eit one

whos e on lyusers are als o itsmaintainers. A go o d test doesnot explo de on fai lurebut prints a

clearand succinct des crip tionofthe sympt omofthe pro blem, andperhaps other relevant

fac ts ab out the context. Ide ally,the maintainer shouldnot need toreadthe sourcecodeto

de ciph eratestfai lure. A go o d test shouldnot giveupafter one fai lurebut shouldtry torep ort

several erro rs in a singlerun, since the pattern offai lures may its elf berevealing.

Theass ertionfunctionbelow comparestwo values, cons tructsagener ic er ror message , and

stopsthe program. It’seasy touse and it’scor rec t,but whenitfai ls, the error message isalmost

us eless. Itdoesnot solvethe hardpro blem ofprovidingagood userint erface.

import (

"fmt"

"strings"

"testing"

)

// A poor assertion function.

func assertEqual(x, y int) {

if x != y {

panic(fmt.Sprintf("%d != %d", x, y))

}

func TestSplit(t *testing.T) {

words := strings.Split("a:b:c", ":")

assertEqual(len(words), 3)

// ...

}

In thissense,ass ertionfunctions suf fer fro m premat ure abstrac tion:bytre ating the fai lureof

this par tic ulartestasameredif ference oftwo int egers,wefor feitthe opp ortunity toprovide

me aningf ul cont ext. Wecan provide a bettermessage bystartingfro m thecon crete det ails,as

in the examplebelow.Onlyoncerep etit ive patternsemerge inagiven testsuite isittimeto

introduce abstrac tions.

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

SECTION 11.2. TEST FUNCTIONS 317

func TestSplit(t *testing.T) {

s, sep := "a:b:c", ":"

words := strings.Split(s, sep)

if got, want := len(words), 3; got != want {

t.Errorf("Split(%q, %q) returned %d words, want %d",

s, sep, got, want)

}

// ...

}

No w thetestrep ortsthe functionthatwas cal le d,its inputs, andthe sig niﬁcanceofthe result;

it explicitlyidentiﬁes the actualvalue andthe exp ectat ion; anditcontinues toexe cut e even if

this ass ertionshouldfai l.Oncewe’ve writt enatestlikethis, the natural next stepisoften not

to deﬁne a functiontoreplace the ent ire if st atement,but toexe cut e thetestinaloopin

which s, sep,and want var y,likethe table-dr iventestof IsPalindrome.

Thepre vious exampledidn’tneed any utilit y func tions,but of cours e that shouldn’tstopus

from int roducingfunctions whenthe y helpmakethe codesimpler.(We’ll look at one such

ut i lit y func tion, reflect.DeepEqual,inSec tion 13.3.) Thekey toagood testistostart by

implementing the con crete beh avior thatyou wantand onlythenuse functions tosimplif y the

co de andeliminaterep etit ion.Bestresults arerarelyobt ained bystartingwit h alibrar y of

abstrac t, generictesting functions.

Exercis e 11.5: Extend TestSplit to use a table ofinp uts andexp ected out puts.

11.2.6. Avoiding Brittle Tests

An app lic ationthatoften fails whenitencount ers new but valid inputs iscal le d bug gy;atest

that spuriou sly fai lswhenasound change was made tothe program iscal le d br ittle.Justasa

bugg y prog ram frust rates its users,abritt letestexasperates its maintainers. Themostbritt le

tests, whichfai l foralmostany change tothe pro duc tioncode, good orbad,are som etimes

called ch angedetec tor or statu s qu o tests, andthe timespent dealingwit h them can quickly

depleteany beneﬁtthe y on ceseeme d to provide.

Wh enafunctionunder testpro duces a comp lex out put suchasalon g st ring, anelaboratedat a

st ruc ture, oraﬁle,it’stempt ing toche ckthatthe out put isexac tly equ altosom e ‘‘go lden’’ value

that was expec ted whenthe testwas writt en. But asthe program evo l ves, par ts of the out put

wi l l li kelychange , prob ably ingood ways, but change non etheless. And it’snot justthe out put;

func tions wit h comp lex inputs often bre akbecause the inp utused in a testisnolon g ervalid.

Theeasiestway toavoid britt letests istoche ckonlythe pro per ties you careabout.Testyour

prog ram’ssimpler andmorestableint erfaces in preferencetoits internal functions.Beselec-

tive inyourass ertions.Don’tche ckfor exac t st ringmatch es, for example, but lookfor relevant

su bst rings thatwill remain unch ange d as the program evo l ves. It’soften wor thwriting a

su bst ant ial functiontodistill a complex out put dow n to its essencesothatass ertions will be

re liable. Eventhoug h that may seemlikealot ofup-f ronteffor t,itcan pay for itself quicklyin

time thatwou ldother wis e be spent ﬁxingspuriou sly fai lingtests.

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

318 CHAPTER 11. TESTING

11.3. Coverage

By its nature, test ing isnever comp lete. Asthe inﬂuential comp uterscientist EdsgerDijkstra

putit, ‘‘Te st ing shows the presence, not the abs ence ofbugs.’’ No quant ity oftests can ever

prov e apackagefre e of bugs. Atbest, the y increase our conﬁdence thatthe packagewor ks

we ll in a wide range ofimp ortantscenar ios.

Thedeg ree towhichatestsuite exercis esthe packageunder testiscal le d thetest’s covera ge.

Coveragecan’tbequant iﬁed direc tly—thedynamics ofall but the most trivialprogramsare

beyond pre cis e me asurement—butthere are heur ist ics thatcan helpusdirec t ourtesting

ef for tstowhere the y aremorelikelytobeuseful.

St atement cov era ge is the simplestand most widely usedofthese heur ist ics. Thestatement

coverageofatestsuite isthe frac tionofsourcestatementsthatare exe cut edatleast oncedur-

ingthe test. Inthissec tion,we’ll use Go’s cover to ol, whichisint egrated int o go test,to

me asure statement cov erageand helpidentify obv iou s gaps in the tests.

Thecodebelow isatable-dr iventestfor the expressionevaluatorwebui ltback in Chapt er7:

gopl.io/ch7/eval

func TestCoverage(t *testing.T) {

var tests = []struct {

input string

env Env

want string // expected error from Parse/Check or result from Eval

}{

{"x % 2", nil, "unexpected '%'"},

{"!true", nil, "unexpected '!'"},

{"log(10)", nil, `unknown function "log"`},

{"sqrt(1, 2)", nil, "call to sqrt has 2 args, want 1"},

{"sqrt(A / pi)", Env{"A": 87616, "pi": math.Pi}, "167"},

{"pow(x, 3) + pow(y, 3)", Env{"x": 9, "y": 10}, "1729"},

{"5 / 9 * (F - 32)", Env{"F": -40}, "-40"},

}

for _, test := range tests {

expr, err := Parse(test.input)

if err == nil {

err = expr.Check(map[Var]bool{})

}

if err != nil {

if err.Error() != test.want {

t.Errorf("%s: got %q, want %q", test.input, err, test.want)

}

continue

}

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

SECTION 11.3. COVERAGE 319

got := fmt.Sprintf("%.6g", expr.Eval(test.env))

if got != test.want {

t.Errorf("%s: %v => %s, want %s",

test.input, test.env, got, test.want)

}

First,let’sche ckthatthe testpasses:

$gotest -v -run=Coverage gopl.io/ch7/eval

=== RUN TestCoverage

--- PASS: TestCoverage (0.00s)

PASS

ok gopl.io/ch7/eval 0.011s

Thiscommand displ ays the usage message ofthe cov eragetool:

$gotool cover

Usage of 'go tool cover':

Given a coverage profile produced by 'go test':

go test -coverprofile=c.out

Open a web browser displaying annotated source code:

go tool cover -html=c.out

...

The go tool commandrunsone ofthe exe cut ables fro m theGotoolchain. Thes e prog rams

live inthe direc tor y $GOROOT/pkg/tool/${GOOS}_${GOARCH}.Thanksto go build,werarely

ne e d to inv oke themdirec tly.

No w we run the testwit h the -coverprofile ﬂag:

$gotest -run=Coverage -coverprofile=c.out gopl.io/ch7/eval

ok gopl.io/ch7/eval 0.032s coverage: 68.5% of statements

This ﬂag enables the col lec tion of cov eragedat a by inst rum ent ing thepro duc tioncode. That

is,itmodiﬁes a copy ofthe sourcecodesothatbeforeeachblo ckofstatementsisexe cut ed, a

boole an var iable isset, wit h on e var iable per blo ck. Justbeforethe modiﬁed program exits, it

wr itesthe value ofeachvar iable tothe speciﬁed log ﬁle c.out andprintsasummar y of the

frac tionofstatementsthatwereexe cut ed. (If all you need isthe summar y,use

go test -cover.)

If go test is run wit h the -covermode=count ﬂag , theins trument ation for eachblo ckincre-

mentsacount erins teadofsetting a boole an. Theresulting log ofexe cut ion countsofeach

blockenables quant itative comparisons bet ween ‘‘hott er’’ blocks, whichare morefre quently

exec ute d, and ‘‘colder’’ on es.

Having gat hered the dat a, we run the cover to ol, whichpro cessesthe log , generates an HTML

report,and opens it in a new brows er window (Figure11.3).

$gotool cover -html=c.out

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

320 CHAPTER 11. TESTING

Figure 11.3. Acov eragerep ort.

Each statement iscolored greenifitwas covere d or red ifitwas not cov ere d.For clarity,we’ve

sh ade d thebackg round ofthe red text. Wecan see immediate lythatnon e of our inputs exer-

cisedthe unary operator Eval method.Ifweadd thisnew testcas e to the table andre-r un the

previous two command s,the unary expressioncodebecom esgre en:

{"-x * -x", eval.Env{"x": 2}, "4"}

Thetwo panic st atementsremain red,how ever. Thisshouldnot besur prising, because these

st atementsare sup pos edtobeunreach able.

Ac hie ving100% statement cov eragesound s li keanoble goal, but it isnot usu allyfeasiblein

prac tice,nor isitlikelytobeagood use ofeffor t.Justbecause a statement isexe cut eddoes

notmeanitisbug-f ree;statementscontainingcomplex expressions mustbeexe cut edmany

timeswit h dif ferentinp uts tocov erthe int erest ing cas es. Some statements, like the panic

st atementsabove , cannever bereach ed. Others, suchasthose thathandleesotericerror s,are

hard toexercis e butrarelyreach edinprac tice.Testing isfundament allyaprag mat ic ende avor,

atrade-off bet weenthe costofwriting tests andthe costoffai lures thatcou ldhavebeen

prevente d by tests. Cov eragetools can helpidentify the weakest spots, but devisinggood test

casesdemands the samerigorou s thin king as programming in general.

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

SECTION 11.4. BENCHMARK FUNCTIONS 321

11.4. Benchmark Functions

Benchmarking isthe prac tice ofmeasuring the per for mance ofaprogram onaﬁxe d

work load. InGo, a benchmark functionlooks like a testfunction, but wit h the Benchmark

preﬁxand a *testing.B parameter thatprovides mostofthe samemet hodsasa*testing.T,

plus a few ext ra re lated toper for mance measurement.Itals o exp oses an integerﬁeld N,which

sp eciﬁes the numberoftimes toper for m theoperat ionbeingmeasure d.

Here’sabenchmark for IsPalindrome that cal lsit N times in a loop.

import "testing"

func BenchmarkIsPalindrome(b *testing.B) {

for i := 0; i < b.N; i++ {

IsPalindrome("A man, a plan, a canal: Panama")

}

We run itwit h thecommand below.Unliketests, bydefau ltnobenchmarksare run. The

argumenttothe -bench ﬂag selec tswhichbenchmarkstorun. Itisaregu lar expression

matchingthe names of Benchmark func tions,wit h adefau ltvalue thatmatch esnon e of them.

The ‘‘.’’ patt ern causesittomatch all benchmarksinthe word package, but since there’sonly

on e, -bench=IsPalindrome wouldhavebeenequivalent.

$cd$GOPATH/src/gopl.io/ch11/word2

$gotest -bench=.

PASS

BenchmarkIsPalindrome-8 1000000 1035 ns/op

ok gopl.io/ch11/word2 2.179s

Thebenchmark name’snumer ic sufﬁx, 8 here , indic ates the value of GOMAXPROCS,whichis

importantfor con cur rentbenchmarks.

Therep ort tel lsusthateachcal l to IsPalindrome to okabout 1.035 micros econd s,averaged

ov er1,000,000 runs. Since the benchmark runner initial lyhas noide a howlon g theoperat ion

takes, itmakes som e initial measurementsusingsmall values of N andthenext rapol atestoa

value large enoug h forastabletimingmeasurement tobemade.

Thereasonthe loopisimp lemente d by the benchmark function, andnot bythe cal lingcodein

thetestdriver, issothatthe benchmark functionhas the opp ortunity toexe cut e anynecessary

on e-t imesetup codeoutside the loopwit houtthisaddingtothe measured time ofeachitera-

tion.Ifthissetup codeisstill per turbingthe results, the testing.B parameter provides met h-

odstostop, resume, and res et thetimer,but these are rarelyneeded.

No w that wehaveabenchmarkand tests, it’seasy totry out ide as formak ingthe program

faster. Perhaps the most obviou s opt imizat ionistomake IsPalindrome’ssecon d lo opstop

ch eck ing atthe midpoint, toavoid doingeachcomparisontwice:

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

322 CHAPTER 11. TESTING

n:=len(letters)/2

for i := 0; i < n; i++ {

if letters[i] != letters[len(letters)-1-i] {

return false

}

return true

But as is often the cas e,anobv iou s opt imizat iondoesn’talways yield the exp ected beneﬁt.

Thisone delivered a mere 4% imp rov ement inone exp eriment.

$gotest -bench=.

PASS

BenchmarkIsPalindrome-8 1000000 992 ns/op

ok gopl.io/ch11/word2 2.093s

Anot her ideaistopre-al locate a sufﬁcientlylarge array for use by letters,rat her than

exp anditbysuccessive cal lsto append.Declaring letters as an array ofthe rig htsize,like

this,

letters := make([]rune, 0, len(s))

for _, r := range s {

if unicode.IsLetter(r) {

letters = append(letters, unicode.ToLower(r))

}

yields animp rov ement ofnearly35%, andthe benchmark runner now rep ortsthe average

ov er2,000,000 iterat ions.

$gotest -bench=.

PASS

BenchmarkIsPalindrome-8 2000000 697 ns/op

ok gopl.io/ch11/word2 1.468s

As thisexampleshows, the fastest program isoften the one thatmakes the fewestmemor y

al location s.The -benchmem command-line ﬂag will include memor y al location statist ics in its

report.Herewecompare the numberofallocat ions beforethe opt imizat ion:

$gotest -bench=. -benchmem

PASS

BenchmarkIsPalindrome 1000000 1026 ns/op 304 B/op 4allocs/op

andafter it:

$gotest -bench=. -benchmem

PASS

BenchmarkIsPalindrome 2000000 807 ns/op 128 B/op 1 allocs/op

Cons olid atingthe allocat ions inasinglecal l to make eliminated 75% ofthe allocat ions and

halved the quant ity ofallocated memor y.

Benchmarks like thistel l us the abs olut e time required for a given operat ion, but inmanyset-

tingsthe int erest ing per for mance question s areabout the re lat ive timings oftwo dif ferent

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

SECTION 11.5. PROFILING 323

op erat ions.For example, ifafunctiontakes 1mstopro cess1,000 elements, how lon g wi l l it

take topro cess10,000 oramillion? Suchcomparisons revealthe asy mpt otic growt h of the

runningtimeofthe function. Another example: whatisthe bestsize for anI/O buf fer?

Benchmarks ofapp lic ationthroughputoverarange ofsizes can helpuscho ose the smallest

buffer thatdeliverssat isfac tor y perfor mance.Athirdexample: whichalgor it hmper for msbest

foragiven job?Benchmarksthatevaluate two dif ferentalgor it hms onthe sameinp utdat a

canoften showthe strengt hsand weaknessesofeachone onimp ortantorrepresent ative

work loads.

Comp arat ive benchmarksare justregu lar code.The y typicallytakethe for m of a single

parameter izedfunction, cal le d from several Benchmark func tions wit h dif ferentvalues, like

this:

func benchmark(b *testing.B, size int) { /* ... */ }

func Benchmark10(b *testing.B) {benchmark(b, 10) }

func Benchmark100(b *testing.B) {benchmark(b, 100) }

func Benchmark1000(b *testing.B) { benchmark(b, 1000) }

Theparameter size,whichspeciﬁes the size ofthe inp ut, varies acrossbenchmarksbut is

cons tantwit hin eachbenchmark.Resist the tempt ation touse the parameter b.N as the inp ut

size.Unlessyou int erpretitasaniterat ioncount for a ﬁxe d-size input, the results ofyour

benchmarkwill bemeaningless.

Patt ernsreveale d by comparat ive benchmarksare par tic ularlyusefuldur ingprogram design,

butwedon’tthrow the benchmarksaway whenthe program iswor king. Asthe program

evolves, orits inputgrows, oritisdeploye d on new operat ingsystems orpro cessors wit h dif-

ferent charac ter ist ics, wecan reuse those benchmarkstorevisit desig n de cisions.

Exercis e 11.6: Wr ite benchmarkstocompare the PopCount implementation inSec tion 2.6.2

with yoursolut ion s to Exercis e 2.4 andExercis e 2.5. Atwhatpoint doesthe table-b ased

approach bre akeven?

Exercis e 11.7: Wr ite benchmarksfor Add, UnionWith,and other met hodsof *IntSet (§6.5)

usinglarge pseudo-randominp uts. How fastcan you makethese met hodsrun? How doesthe

ch oice ofwordsize affe ctper for mance? How fastis IntSet comp are d to a set implementation

basedonthe bui lt-in map typ e?

11.5. Pro filing

Benchmarks areusefulfor measuring the per for mance ofspeciﬁcoperat ions,but whenwe’re

tr yingtomakeaslowprogram faster, weoften havenoide a wheretobeg in. Ev ery program-

merknows Don ald Knu th’saph orism aboutpremature opt imizat ion, whichapp eared in

‘‘St ruc tured Programmingwit h go toStatements’’ in 1974. Although often misinterprete d to

me anper for mance doesn’tmatter, inits originalcontext wecan discer n adif ferentmeaning:

Thereisnodou bt that the grai l of efﬁciency leads toabu se. Programmerswaste

enor mou s amountsoftimethin kingabout,orwor rying about,the speed ofnon critical

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

324 CHAPTER 11. TESTING

partsoftheir programs, andthese att emp tsatefﬁciency actuallyhaveastron g negat ive

impact when debuggingand maintenanceare con sidered.We sh oul d forgetabout

smal l efﬁciencies, say about97% ofthe time: premature opt imizat ionisthe rootofall

evil.

Ye tweshouldnot passupour opp ortunities in thatcritical3%. A go o d prog rammer

wi l l notbelulled int o comp lacency bysuchreasoning, hewill bewis e to lookcaref ully

at the criticalcode; but only af ter that code has beenidentiﬁed.Itisoften a mist ake to

make a prior i judg ments aboutwhatpar ts of a prog ram arereallycritical, since the

univers alexp erience ofprogrammerswho havebeenusingmeasurement tools has

been thattheir intuit ive guessesfai l.

Wh enwewishtolookcaref ullyatthe speed ofour programs, the besttechnique for identify-

ingthe criticalcodeis proﬁling.Proﬁlingisanaut omated appro ach to per for mance measure-

ment bas edonsamplinganumberofproﬁle events during exe cut ion,thenext rapol ating fro m

them dur ingapost-pro cessingstep; the resulting statist icalsummar y is cal le d a proﬁle.

Go sup por tsmanykinds ofproﬁling, eachcon cer ned wit h adif ferentasp ect ofper for mance,

butall oftheminv olverecordingasequence ofeventsofint erest,eachofwhichhas an accom-

pany ing stack trace—thestack offunctioncal lsactiveatthe moment ofthe event.The

go test to olhas bui lt-in sup por t forseveral kinds ofproﬁling.

A CPU proﬁle identiﬁes the functions whose exe cut ion requires the most CPU time. The cur-

rent lyrunningthreadoneachCPU isint err upt edper iodic ally bythe operat ingsystemevery

fe w mi l lisecond s,wit h each int err upt ion recordingone proﬁle event beforenor mal exec ution

resumes.

A he approﬁle identiﬁes the statementsresponsible for allocat ingthe most memor y.The

proﬁlinglibrar y samp les cal lstothe int ernal memor y al location routinessothatonaverage,

on e proﬁle event isrecorde d per512KB ofallocated memor y.

A bl o cki n g proﬁle identiﬁes the operat ions responsible for blo cking goro utinesthe lon g est,

such assystemcal ls, channel sends and receives, andacquisition s of locks. Theproﬁling

librar y re cords anevent every timeagoroutine isblo cke d by one ofthese operat ions.

Gat her ingaproﬁle for codeunder testisaseasy as enablingone ofthe ﬂags below.Becaref ul

when usingmorethanone ﬂag at a time, how ever: the machiner y forgat her ingone kindof

proﬁle may ske w theresults ofothers.

$gotest -cpuprofile=cpu.out

$gotest -blockprofile=block.out

$gotest -memprofile=mem.out

It’seasy toadd proﬁlingsup por t to non-testprogramstoo,thoug h thedet ails ofhow wedo

that varybet weenshort-live d command-linetools and lon g-r unningser ver applic ations.

Proﬁlingisesp eci ally usefulinlon g-r unningapp lic ations,sothe Goruntime’sproﬁling

fe aturescan beenabled under programmer cont rol usingthe runtime API.

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

SECTION 11.5. PROFILING 325

Once we’ve gat hered a proﬁle,weneed toanalyze itusingthe pprof to ol. Thisisastand ard

part ofthe Godistr ibution,but since it’snot aneverydaytool, it’saccessedindirec tly using

go tool pprof.Ithas dozensoffeaturesand opt ion s,but basic use requires onlytwo argu-

ments, the exe cut ablethatpro duce d theproﬁle andthe proﬁle log.

To m akeproﬁlingefﬁcientand tosavespace,the log does not include functionnames; instead,

func tions are identiﬁed bytheir addresses. Thismeans that pprof ne e dsthe exe cut ablein

order tomakesense ofthe log . Although go test usuallydis cards the testexe cut ableoncethe

test iscomplete, whenproﬁlingisenabled itsaves the exe cut ableas foo.test,where foo is

thenameofthe teste d package.

Thecommand s belowshowhow togat her anddispl ayasimpleCPU proﬁle.We’ve selec ted

on e of the benchmarksfro m the net/http package. Itisusu allybettertoproﬁle speciﬁc

benchmarks thathavebeencon str ucted toberepresent ative ofwor klo adsone cares about.

Benchmarking testcas es is almostnever represent ative , whichiswhy wedis abled themby

usingthe ﬁlt er -run=NONE.

$gotest -run=NONE -bench=ClientServerParallelTLS64 \

-cpuprofile=cpu.log net/http

PASS

BenchmarkClientServerParallelTLS64-8 1000

3141325 ns/op 143010 B/op 1747 allocs/op

ok net/http 3.395s

$gotool pprof -text -nodecount=10 ./http.test cpu.log

2570ms of 3590ms total (71.59%)

Dropped 129 nodes (cum <= 17.95ms)

Showing top 10 nodes out of 166 (cum >= 60ms)

flat flat% sum% cum cum%

1730ms 48.19% 48.19% 1750ms 48.75% crypto/elliptic.p256ReduceDegree

230ms 6.41% 54.60% 250ms 6.96% crypto/elliptic.p256Diff

120ms 3.34% 57.94% 120ms 3.34% math/big.addMulVVW

110ms 3.06% 61.00% 110ms 3.06% syscall.Syscall

90ms 2.51% 63.51% 1130ms 31.48% crypto/elliptic.p256Square

70ms 1.95% 65.46% 120ms 3.34% runtime.scanobject

60ms 1.67% 67.13% 830ms 23.12% crypto/elliptic.p256Mul

60ms 1.67% 68.80% 190ms 5.29% math/big.nat.montgomery

50ms 1.39% 70.19% 50ms 1.39% crypto/elliptic.p256ReduceCarry

50ms 1.39% 71.59% 60ms 1.67% crypto/elliptic.p256Sum

The -text ﬂag speciﬁes the out put for mat, in thiscas e,atextualtable wit h on e rowper func-

tion,sor ted sothe ‘‘hott est’’ func tions—thos e that cons ume the most CPU cyc les—appear

ﬁrs t.The -nodecount=10 ﬂag limits the resultto10rows. For gross per for mance pro blems,

this textualfor mat may beenoug h to pinpointthe cause.

Thisproﬁle tel lsusthatellip tic-c urve crypt ography isimp ortanttothe per for mance ofthis

partic ularHTTPS benchmark.Bycontrast, if a proﬁle isdominated bymemor y al location

func tions fro m the runtime package, reducingmemor y cons ump tionmay beawor thw hile

opt imizat ion.

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

326 CHAPTER 11. TESTING

Fo r more subtlepro blems, you may bebetteroff usingone of pprof’sgraphic al displays.

Thes e re quireGraphViz, whichcan bedow nlo ade d from www.graphviz.org.The -web ﬂag

then rendersadirec ted graphofthe functions ofthe program, annot ate d by their CPU proﬁle

numb ers and colored toindic atethe hott est functions.

We’ve onlyscratch edthe sur face of Go’sproﬁlingtools here. Toﬁnd out more, readthe

‘‘ProﬁlingGoPrograms’’ ar ticle onthe GoBlog .

11.6. Example Functions

Thethirdkindoffunctiontre ate d sp eci ally by go test is anexamplefunction, one whose

name startswit h Example.Ithas neither parametersnor results. Here’sanexamplefunction

for IsPalindrome:

func ExampleIsPalindrome() {

fmt.Println(IsPalindrome("A man, a plan, a canal: Panama"))

fmt.Println(IsPalindrome("palindrome"))

// Output:

// true

// false

}

Examplefunctions ser vethree pur pos es. Theprimary one isdocumentation:agood example

canbeamoresuccinct orint uit ive way toconve y thebeh avior of a librar y func tionthanits

pros e des crip tion, especi ally whenusedasareminder orquickreference.Anexamplecan

also demon stratethe int erac tionbet weenseveral typ es andfunctions belon gingtoone API,

whereasprose documentation mustalways beatt ach edtoone place,likeatyp e or function

de clarat ionorthe packageasawhole.And unlikeexamples wit hin comments, examplefunc-

tion s arerealGocode, subjec t to compi le-t imeche cking , so the y don’tbecom e st ale as the

co de evolves.

Basedonthe sufﬁxofthe Example func tion, the web-b ased documentation ser ver godoc

asso ciatesexamplefunctions wit h thefunctionorpackagethe y exemplif y,so ExampleIs-

Palindrome wouldbeshown with thedocumentation for the IsPalindrome func tion, andan

examplefunctioncal le d ju st Example wouldbeass oci ated wit h the word package as a whole.

Thesecon d purpos e is thatexamples areexe cut abletests run by go test.Ifthe examplefunc-

tion containsaﬁnal // Output: commentlikethe one above , thetestdriverwill exe cut e the

func tionand che ckthatwhatitprint edtoits stand ard out put match esthe text wit hin the

comment.

Thethirdpur pos e of anexampleishands-onexp erimentation.The godoc server at

golang.org us esthe GoPlayg round tolet the usereditand run eachexamplefunctionfro m

within a web brows er,asshown inFigure11.4. Thisisoften the fastest way toget a feelfor a

partic ularfunctionorlangu agefeature .

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

SECTION 11.6. EXAMPLE FUNCTIONS 327

Figure 11.4. An int erac tiveexampleof strings.Join in godoc.

Theﬁnaltwo chapt ers ofthe bookexaminethe reflect and unsafe packages, whichfew Go

prog rammersregu larly use—andevenfewer ne e d to use.Ifyou haven’twritt enany subst ant ial

Go programsyet, now wou ldbeagood time to do that.

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

This page intentionally left blank

From the Library of YIGUANG HU

ptg16105617

Re flection

Go provides a mechanism toupdatevar iables andins pec t their values at run time, to cal l their

methods, andtoapp l y theoperat ions int rinsic totheir represent ation,all wit houtknowing

their typ es at comp ile time. Thismechanism iscal le d reﬂec tion.Reﬂec tion als o lets ustre at

typesthems elves as ﬁrs t-class values.

In thischapt er, we’ll explore Go’sreﬂec tion featurestosee how the y increase the expres-

siveness ofthe langu age, and inpar tic ularhow the y arecruci al to the imp lementation of two

importantAPIs: str ing for matting provide d by fmt,and pro tocol encodingprovide d by pack-

ages like encoding/json and encoding/xml.Reﬂec tion isals o essent ial tothe templ ate

me chanism provide d by the text/template and html/template packages wesaw in

Section4.6. How ever, reﬂec tion iscomplex toreasonabout and notfor casualuse,so

although these packages areimp lemente d usingreﬂec tion,the y do not exp osereﬂec tion in

their own APIs.

12.1. Why Reflection?

Sometimesweneed towrite a functioncap able ofdealingunifor mly wit h values oftyp es that

don’tsat isf y acommonint erface,don’thaveaknown represent ation,ordon’texist atthe time

we desig n thefunction—or evenall three.

Afami liar exampleisthe for matting log ic within fmt.Fprintf,whichcan usefullyprint an

arbit rar y value ofany typ e,evenauser-deﬁne d on e.Let’stry toimp lementafunctionlikeit

usingwhatweknowalready.For simplicity,our functionwill acceptone argumentand will

return theresult as a str ing like fmt.Sprint do es, so we’ll cal l it Sprint.

We start wit h atyp e sw itch thattests whether the argumentdeﬁnesaString method,and cal l

it ifso. Wethenadd swit c h casesthattestthe value’sdynamic typ e against eachofthe basic

329

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

330 CHAPTER 12. REFLECTION

types—string, int, bool,and soon—andper for m theappro priatefor matting operat ionin

each cas e.

func Sprint(x interface{}) string {

type stringer interface {

String() string

}

switch x := x.(type) {

case stringer:

return x.String()

case string:

return x

case int:

return strconv.Itoa(x)

// ...similar cases for int16, uint32, and so on...

case bool:

if x {

return "true"

}

return "false"

default:

// array, chan, func, map, pointer, slice, struct

return "???"

}

Buthow dowedealwit h ot her typ es, li ke []float64, map[string][]string,and soon? We

couldadd morecas es, butthe numberofsuchtyp es is inﬁnite. And whatabout named typ es,

li ke url.Values?Evenifthe typ e sw itch had a cas e forits underly ing typ e

map[string][]string,itwou ldn’tmatch url.Values becaus e thetwo typ es arenot iden-

tical, and the typ e sw itch cannot include a cas e foreachtyp e li ke url.Values becaus e that

wouldrequirethislibrar y to dep enduponits clients.

Wi thout a way toins pec t therepresent ation of values ofunknown types, wequicklyget stuck.

Wh atweneed isreﬂec tion.

12.2. reflect.Type and reflect.Value

Reﬂec tion isprovide d by the reflect package. Itdeﬁnestwo imp ortanttyp es, Type and

Value.AType repres ents a Gotyp e.Itisanint erface wit h many met hodsfor dis criminat ing

amon g typesand ins pec ting their comp onents, like the ﬁelds ofastr uct orthe parametersofa

func tion. Thesoleimp lementation of reflect.Type is the typ e des crip tor (§7.5), the same

entity thatidentiﬁes the dynamic typ e of anint erface value.

The reflect.TypeOf func tionaccepts any interface{} andretur nsits dynamic typ e as a

reflect.Type:

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

SECTION 12.2. REFLECT.TYPE AND REFLECT.V ALUE331

t:=reflect.TypeOf(3) // areflect.Type

fmt.Println(t.String()) // "int"

fmt.Println(t) // "int"

The TypeOf(3) call above assig nsthe value 3 to the interface{} parameter.Recal l from

Section7.5 thatanassig nmentfro m acon crete value toanint erface typ e perfor msanimp licit

interface conv ersion, whichcre atesanint erface value consisting oftwo components: its

dy namic type is the operand’styp e (int)and its dy namic valu e is the operand’svalue (3).

Becaus e reflect.TypeOf returns an int erface value’sdynamic typ e,italways retur nsacon-

cretetyp e.So, for example, the codebelow prints "*os.File",not "io.Writer".Later,we

wi l l seethat reflect.Type is cap able ofrepresent ing int erface typ es to o.

var w io.Writer = os.Stdout

fmt.Println(reflect.TypeOf(w)) // "*os.File"

No tice that reflect.Type satisﬁes fmt.Stringer.Because print ing the dynamic typ e of an

interface value isusefulfor debug gingand log ging, fmt.Printf prov ides a shorthand, %T,that

us es reflect.TypeOf internal ly:

fmt.Printf("%T\n", 3) // "int"

Theother importanttyp e in the reflect packageis Value.Areflect.Value canholda

value ofany typ e.The reflect.ValueOf func tionaccepts any interface{} andretur nsa

reflect.Value cont ainingthe int erface’sdynamic value.Aswit h reflect.TypeOf,the

resu lts of reflect.ValueOf arealways concrete, but a reflect.Value canholdint erface val-

ues too.

v:=reflect.ValueOf(3) // a reflect.Value

fmt.Println(v) // "3"

fmt.Printf("%v\n", v) // "3"

fmt.Println(v.String()) // NOTE: "<int Value>"

Like reflect.Type, reflect.Value also sat isﬁes fmt.Stringer,but unlessthe Value holds

astr ing , theresultofthe String method reveals onlythe typ e.Ins tead, use the fmt package’s

%v verb,whichtre ats reflect.Valuesspeci ally.

Callingthe Type method onaValue returnsits typ e as a reflect.Type:

t:=v.Type() // areflect.Type

fmt.Println(t.String()) // "int"

Theinv ers e op erat ionto reflect.ValueOf is the reflect.Value.Interface method.It

returnsan interface{} holdingthe samecon crete value as the reflect.Value:

v:=reflect.ValueOf(3) // a reflect.Value

x:=v.Interface() // an interface{}

i:=x.(int) // an int

fmt.Printf("%d\n", i) // "3"

A reflect.Value andan interface{} canbot h hold arbit rar y values. Thedif ference isthat

an empt y interface hides the represent ation and int rinsic operat ions ofthe value itholds and

exp oses non e of itsmet hods, sounlessweknowits dynamic typ e anduse a typ e assertionto

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

332 CHAPTER 12. REFLECTION

peer inside it(as wedid abov e), there islit tle wecan do tothe value wit hin. Incontrast, a

Value hasmanymet hodsfor ins pec ting its cont ents, regardlessofits typ e.Let’suse themfor

oursecon d attemp t at a general for matting function, whichwe’ll cal l format.Any.

In ste adofatyp e sw itch,weuse reflect.Value’s Kind method todis criminatethe cas es.

Although there are inﬁnite lymanytyp es, thereare onlyaﬁnite numberof ki nds of typ e: the

basic typ es Bool, String,and all the numbers;the aggregatetyp es Array and Struct;the ref-

erence typ es Chan, Func, Ptr, Slice,and Map; Interface types; and ﬁnally Invalid,meaning

no value at all. (Thezerovalue ofareflect.Value haskind Invalid.)

gopl.io/ch12/format

package format

import (

"reflect"

"strconv"

)

// Any formats any value as a string.

func Any(value interface{}) string {

return formatAtom(reflect.ValueOf(value))

}

// formatAtom formats a value without inspecting its internal structure.

func formatAtom(v reflect.Value) string {

switch v.Kind() {

case reflect.Invalid:

return "invalid"

case reflect.Int, reflect.Int8, reflect.Int16,

reflect.Int32, reflect.Int64:

return strconv.FormatInt(v.Int(), 10)

case reflect.Uint, reflect.Uint8, reflect.Uint16,

reflect.Uint32, reflect.Uint64, reflect.Uintptr:

return strconv.FormatUint(v.Uint(), 10)

// ...floating-point and complex cases omitted for brevity...

case reflect.Bool:

return strconv.FormatBool(v.Bool())

case reflect.String:

return strconv.Quote(v.String())

case reflect.Chan, reflect.Func, reflect.Ptr, reflect.Slice, reflect.Map:

return v.Type().String() + " 0x" +

strconv.FormatUint(uint64(v.Pointer()), 16)

default: // reflect.Array, reflect.Struct, reflect.Interface

return v.Type().String() + " value"

}

So far,our functiontre ats eachvalue as an indivisiblethingwit h no int ernal str ucture—hence

formatAtom.For aggregatetyp es (str uctsand arrays) andint erfaces itprintsonlythe ty pe of

thevalue,and for reference typ es (channel s,functions,point ers,slices, andmaps), itprintsthe

type and the reference addressinhexade cimal.Thisislessthanide al butstill a maj or

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

SECTION 12.3. DISPLAY,ARECURSIVE VALUE PRINTER 333

improv ement,and since Kind is con cer ned onlywit h theunderly ing represent ation, for-

mat.Any worksfor named typ es to o.For example:

var x int64 = 1

var d time.Duration = 1 * time.Nanosecond

fmt.Println(format.Any(x)) // "1"

fmt.Println(format.Any(d)) // "1"

fmt.Println(format.Any([]int64{x})) // "[]int64 0x8202b87b0"

fmt.Println(format.Any([]time.Duration{d})) // "[]time.Duration 0x8202b87e0"

12.3. Display,aRecursiveValue Printer

Next we’ll takealookathow toimp rov e thedispl ayofcomposite typ es. Rather thantry to

copy fmt.Sprint exac tly,we’ll bui ld adebug gingutilit y func tioncal le d Display that, given

an arbit rar ily complex value x,printsthe completestr uctureofthatvalue,lab elingeachele-

ment wit h thepat h by whichitwas found.Let’sstart wit h an example.

e, _ := eval.Parse("sqrt(A / pi)")

Display("e", e)

In the cal l ab ove , theargumentto Display is a syntaxtre e from the expressionevaluatorin

Section7.9. Theout put of Display is shown below :

Display e (eval.call):

e.fn = "sqrt"

e.args[0].type = eval.binary

e.args[0].value.op = 47

e.args[0].value.x.type = eval.Var

e.args[0].value.x.value = "A"

e.args[0].value.y.type = eval.Var

e.args[0].value.y.value = "pi"

Wh ere possible, you shouldavoid exposingreﬂec tion inthe API ofapackage. We’ll deﬁne an

unexp orted function display to dothe realwor k of the rec ursion, andexp ort Display,asim-

plewrapperaro und itthataccepts an interface{} parameter :

gopl.io/ch12/display

func Display(name string, x interface{}) {

fmt.Printf("Display %s (%T):\n", name, x)

display(name, reflect.ValueOf(x))

}

In display,we’ll use the formatAtom func tionwedeﬁne d earlier toprint elementary values—

basic typ es, func tions,and channel s—butwe’ll use the methodsof reflect.Value to rec ur-

sive lydispl ayeachcomponent ofamorecomplex typ e.Asthe rec ursiondes cends,the path

st ring, whichinitial lydes crib esthe startingvalue (for ins tance, "e"), will beaug mente d to

indic atehow wereach edthe cur rentvalue (for ins tance, "e.args[0].value").

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

334 CHAPTER 12. REFLECTION

Sincewe’re nolon g erpretendingtoimp lement fmt.Sprint,wewill use the fmt packageto

ke epour exampleshort.

func display(path string, v reflect.Value) {

switch v.Kind() {

case reflect.Invalid:

fmt.Printf("%s = invalid\n", path)

case reflect.Slice, reflect.Array:

for i := 0; i < v.Len(); i++ {

display(fmt.Sprintf("%s[%d]", path, i), v.Index(i))

}

case reflect.Struct:

for i := 0; i < v.NumField(); i++ {

fieldPath := fmt.Sprintf("%s.%s", path, v.Type().Field(i).Name)

display(fieldPath, v.Field(i))

}

case reflect.Map:

for _, key := range v.MapKeys() {

display(fmt.Sprintf("%s[%s]", path,

formatAtom(key)), v.MapIndex(key))

}

case reflect.Ptr:

if v.IsNil() {

fmt.Printf("%s = nil\n", path)

}else {

display(fmt.Sprintf("(*%s)", path), v.Elem())

}

case reflect.Interface:

if v.IsNil() {

fmt.Printf("%s = nil\n", path)

}else {

fmt.Printf("%s.type = %s\n", path, v.Elem().Type())

display(path+".value", v.Elem())

}

default: // basic types, channels, funcs

fmt.Printf("%s = %s\n", path, formatAtom(v))

}

Let’sdis cussthe cas es in order.

Slices and array s: Thelog ic is the samefor bot h.The Len method retur nsthe numberofele-

mentsofaslice orarray value,and Index(i) retr ieves the element at index i,als o as a

reflect.Value;itpanics if i is out of bound s.These are analogou s to the bui lt-in len(a)

and a[i] op erat ions onsequences. The display func tionrec ursivelyinv okesits elf oneach

elementofthe sequence,app endingthe subscript not ation "[i]" to the pat h.

Although reflect.Value hasmanymet hods, onlyafew are safetocal l on any given value.

Fo r example, the Index method may becal le d on values ofkind Slice, Array,or String,but

panics for any other kind.

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

SECTION 12.3. DISPLAY,ARECURSIVE VALUE PRINTER 335

Structs: The NumField method rep ortsthe numberofﬁelds inthe str uct,and Field(i)

returnsthe value ofthe i-t h ﬁeldasareflect.Value.The listofﬁelds includes ones

prom ote d from anony mou s ﬁelds.Toapp end the ﬁeldselec tor not ation ".f" to the pat h, we

mu stobt ain the reflect.Type of the str uct and accessthe nameofits i-t h ﬁeld.

Ma ps: The MapKeys method retur nsaslice of reflect.Values, one per map key.Asusu al

when iterat ingoveramap,the order isundeﬁne d. MapIndex(key) returnsthe value cor-

resp ondingto key.Weapp end the subscript not ation "[key]" to the pat h. (We’re cut tinga

corner here. The typ e of a map key isn’trestr icted tothe typ es formatAtom hand les best;

ar rays, st ruc ts, andint erfaces can also bevalid map keys. Extendingthiscas e to print the key

in full isExercis e 12.1.)

Po int ers: The Elem method retur nsthe var iable point edtobyapoint er, again as a

reflect.Value.Thisoperat ionwou ldbesafeevenifthe point ervalue is nil,inwhichcas e

theresultwou ldhavekind Invalid,but weuse IsNil to detec t ni l pointers explicitlysowe

canprint a moreappro priatemessage . We preﬁxthe pat h with a "*" andparenthesize itto

avoidambiguity.

In ter fac es: Again, weuse IsNil to test whether the int erface isnil,and ifnot,weret rie veits

dy namic value using v.Elem() andprint its typ e andvalue.

No w that our Display func tioniscomplete, let’sput it towor k.The Movie type belowisa

slig htvar iat iononthe one inSec tion 4.5:

type Movie struct {

Title, Subtitle string

Year int

Color bool

Actor map[string]string

Oscars []string

Sequel *string

}

Let’sdeclare a value ofthistyp e andsee what Display do es with it:

strangelove := Movie{

Title: "Dr. Strangelove",

Subtitle: "How I Learned to Stop Worrying and Love the Bomb",

Year: 1964,

Color: false,

Actor: map[string]string{

"Dr. Strangelove": "Peter Sellers",

"Grp. Capt. Lionel Mandrake": "Peter Sellers",

"Pres. Merkin Muffley": "Peter Sellers",

"Gen. Buck Turgidson": "George C. Scott",

"Brig. Gen. Jack D. Ripper": "Sterling Hayden",

`Maj. T.J. "King" Kong`:"Slim Pickens",

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

336 CHAPTER 12. REFLECTION

Oscars: []string{

"Best Actor (Nomin.)",

"Best Adapted Screenplay (Nomin.)",

"Best Director (Nomin.)",

"Best Picture (Nomin.)",

}

Thecal l Display("strangelove", strangelove) pr ints:

Display strangelove (display.Movie):

strangelove.Title = "Dr. Strangelove"

strangelove.Subtitle = "How I Learned to Stop Worrying and Love the Bomb"

strangelove.Year = 1964

strangelove.Color = false

strangelove.Actor["Gen. Buck Turgidson"] = "George C. Scott"

strangelove.Actor["Brig. Gen. Jack D. Ripper"] = "Sterling Hayden"

strangelove.Actor["Maj. T.J. \"King\" Kong"] = "Slim Pickens"

strangelove.Actor["Dr. Strangelove"] = "Peter Sellers"

strangelove.Actor["Grp. Capt. Lionel Mandrake"] = "Peter Sellers"

strangelove.Actor["Pres. Merkin Muffley"] = "Peter Sellers"

strangelove.Oscars[0] = "Best Actor (Nomin.)"

strangelove.Oscars[1] = "Best Adapted Screenplay (Nomin.)"

strangelove.Oscars[2] = "Best Director (Nomin.)"

strangelove.Oscars[3] = "Best Picture (Nomin.)"

strangelove.Sequel = nil

We can use Display to displ aythe int ernalsoflibrar y types, suchas *os.File:

Display("os.Stderr", os.Stderr)

// Output:

// Display os.Stderr (*os.File):

// (*(*os.Stderr).file).fd = 2

// (*(*os.Stderr).file).name = "/dev/stderr"

// (*(*os.Stderr).file).nepipe = 0

No tice thatevenunexp orted ﬁelds are visible toreﬂec tion.Bewarethatthe par tic ularout put

of thisexamplemay varyacrossplatfor msand may change overtimeaslibrar ies evolve.

(Thos e ﬁelds are privatefor a reason!) Wecan evenapp l y Display to a reflect.Value and

watch ittraverse the int ernal represent ation of the typ e des crip tor for *os.File.The out put

of the cal l Display("rV", reflect.ValueOf(os.Stderr)) is shown below,thoug h of cours e

your mile agemay vary:

Display rV (reflect.Value):

(*rV.typ).size = 8

(*rV.typ).hash = 871609668

(*rV.typ).align = 8

(*rV.typ).fieldAlign = 8

(*rV.typ).kind = 22

(*(*rV.typ).string) = "*os.File"

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

SECTION 12.3. DISPLAY,ARECURSIVE VALUE PRINTER 337

(*(*(*rV.typ).uncommonType).methods[0].name) = "Chdir"

(*(*(*(*rV.typ).uncommonType).methods[0].mtyp).string) = "func() error"

(*(*(*(*rV.typ).uncommonType).methods[0].typ).string) = "func(*os.File) error"

...

Observethe dif ference bet weenthese two examples:

var i interface{} = 3

Display("i", i)

// Output:

// Display i (int):

// i = 3

Display("&i", &i)

// Output:

// Display &i (*interface {}):

// (*&i).type = int

// (*&i).value = 3

In the ﬁrs t example, Display calls reflect.ValueOf(i),whichretur nsavalue ofkind Int.

As wemention edinSec tion 12.2, reflect.ValueOf always retur nsaValue of a con crete typ e

since itext racts the contents of an int erface value.

In the secon d example, Display calls reflect.ValueOf(&i),whichretur nsapoint erto i,of

kind Ptr.The switch cas e for Ptr calls Elem on thisvalue,whichretur nsaValue repres enting

the variab le i itself,ofkind Interface.AValue obtained indirec tly,likethisone,may rep-

resent any value at all, includingint erfaces. The display func tioncal lsits elf rec ursivelyand

this time, itprintssep aratecomponentsfor the int erface’sdynamic typ e andvalue.

As cur rentlyimp lemente d, Display wi l l ne ver ter minateifitencount ers a cyc leinthe obj e ct

graph, such as thislin ked listthateats its own tai l:

// a struct that points to itself

type Cycle struct{ Value int; Tail *Cycle }

var c Cycle

c=Cycle{42, &c}

Display("c", c)

Display pr intsthisever-grow ing exp ansion:

Display c (display.Cycle):

c.Value = 42

(*c.Tail).Value = 42

(*(*c.Tail).Tail).Value = 42

(*(*(*c.Tail).Tail).Tail).Value = 42

...

ad infinitum

...

Many Goprogramscontain at leastsom e cyclic dat a. Making Display ro bustagainst such

cycles istrick y,requir ingaddition albookkeeping torecordthe set ofreferences thathavebeen

fo llowe d so far ; it iscostlytoo.Ageneral solut ion requires unsafe language features, as we

wi l l see in Sec tion 13.3.

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

338 CHAPTER 12. REFLECTION

Cy cles pos e lessofapro blem for fmt.Sprint becaus e it rarelytries toprint the complete

st ruc ture. For example, whenitencount ers a point er, itbre aks the rec ursionbyprint ing the

pointer’snumer ic value.Itcan get stuck tryingtoprint a slice ormap thatcontainsits elf as an

element, but suchrarecas es do not war rantthe con siderable extra tro ubleofhandlingcyc les.

Exercis e 12.1: Extend Display so thatitcan displaymaps whose keysare str uctsorarrays.

Exercis e 12.2: Make display safe touse oncyc lic dat a st ruc tures byboundingthe numberof

steps ittakes beforeabandoningthe rec ursion. (InSec tion 13.3, we’ll see another way to

detec t cycles.)

12.4. Example: Encoding S-Expressions

Display is a debug gingroutine for displ aying str uctured dat a, butit’snot far short ofbeing

able toencodeor marshal arbit rar y Go obj e cts as messagesinapor table not ation suitablefor

inter-processcommunic ation.

As wesaw in Sec tion 4.5, Go’sstand ard librar y supp ortsavar ietyoffor mats, includingJSON,

XML, andASN.1. Another not ation thatisstill widely usedis S-expre ssi ons,the syntaxof

Lisp.Unlikethe other not ation s,S-expressions are not sup por ted bythe Gostand ard librar y,

notleast because the y have nounivers allyaccepte d deﬁnition,despit e several attemp tsatstan-

dardizat ionand the existence ofmanyimp lementation s.

In thissec tion,we’ll deﬁne a packagethatencodes arbit rar y Go obj e cts usinganS-expression

notation thatsup por tsthe fol low ing con str ucts:

42 integer

"hello" string (with Go-style quotation)

foo symbol (an unquoted name)

(1 2 3) list (zero or more items enclosed in parentheses)

Booleans are tradition allyencoded usingthe symbol t fortrue, and the emp tylist () or the

sy mbol nil forfalse,but for simplicity,our implementation ignores them. Itals o ig nores

ch annel s andfunctions,since their state isopaquetoreﬂec tion.And itignores realand com-

plex ﬂoating-p ointnumbers and int erfaces. Addingsup por t forthemisExercis e 12.3.

We’ll encodethe typ es of GousingS-expressions asfol lows. Int egers and str ingsare encoded

in the obv iou s way.Nil values areencoded asthe symbol nil.Arrays andslices areencoded

usinglistnot ation.

St ruc ts areencoded asalistofﬁeldbindings, each ﬁeldbindingbeingatwo-elementlist

whos e ﬁrs t element(asymbol) isthe ﬁeldnameand whose second elementisthe ﬁeldvalue.

Maps too are encoded asalistofpairs, wit h each pair beingthe key and value ofone map

entr y.Tradition ally, S-expressions represent lists ofkey/value pairsusingasingle cons cell

(key . value) foreachpair,rat her thanatwo-elementlist, but tosimplif y thedecodingwe’ll

ig noredot ted listnot ation.

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

SECTION 12.4. EXAMPLE: ENCODING S-EXPRESSIONS 339

Enco dingisdon e by a singlerec ursivefunction, encode,shown below.Its str uctureisess en-

tial lythe same as thatof Display in the pre vious sec tion:

gopl.io/ch12/sexpr

func encode(buf *bytes.Buffer, v reflect.Value) error {

switch v.Kind() {

case reflect.Invalid:

buf.WriteString("nil")

case reflect.Int, reflect.Int8, reflect.Int16,

reflect.Int32, reflect.Int64:

fmt.Fprintf(buf, "%d", v.Int())

case reflect.Uint, reflect.Uint8, reflect.Uint16,

reflect.Uint32, reflect.Uint64, reflect.Uintptr:

fmt.Fprintf(buf, "%d", v.Uint())

case reflect.String:

fmt.Fprintf(buf, "%q", v.String())

case reflect.Ptr:

return encode(buf, v.Elem())

case reflect.Array, reflect.Slice: // (value ...)

buf.WriteByte('(')

for i := 0; i < v.Len(); i++ {

if i > 0 {

buf.WriteByte('')

}

if err := encode(buf, v.Index(i)); err != nil {

return err

}

buf.WriteByte(')')

case reflect.Struct: // ((name value) ...)

buf.WriteByte('(')

for i := 0; i < v.NumField(); i++ {

if i > 0 {

buf.WriteByte('')

}

fmt.Fprintf(buf, "(%s ", v.Type().Field(i).Name)

if err := encode(buf, v.Field(i)); err != nil {

return err

}

buf.WriteByte(')')

}

buf.WriteByte(')')

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

340 CHAPTER 12. REFLECTION

case reflect.Map: // ((key value) ...)

buf.WriteByte('(')

for i, key := range v.MapKeys() {

if i > 0 {

buf.WriteByte('')

}

buf.WriteByte('(')

if err := encode(buf, key); err != nil {

return err

}

buf.WriteByte('')

if err := encode(buf, v.MapIndex(key)); err != nil {

return err

}

buf.WriteByte(')')

}

buf.WriteByte(')')

default: // float, complex, bool, chan, func, interface

return fmt.Errorf("unsupported type: %s", v.Type())

}

return nil

}

The Marshal func tionwraps the encoder in an APIsimi lar tothose ofthe other encod-

ing/... packages:

// Marshal encodes a Go value in S-expression form.

func Marshal(v interface{}) ([]byte, error) {

var buf bytes.Buffer

if err := encode(&buf, reflect.ValueOf(v)); err != nil {

return nil, err

}

return buf.Bytes(), nil

}

Here’sthe out put of Marshal applie d to the strangelove var iable fro m Section12.3:

((Title "Dr. Strangelove") (Subtitle "How I Learned to Stop Worrying and Lo

ve the Bomb") (Year 1964) (Actor (("Grp. Capt. Lionel Mandrake" "Peter Sell

ers") ("Pres. Merkin Muffley" "Peter Sellers") ("Gen. Buck Turgidson" "Geor

ge C. Scott") ("Brig. Gen. Jack D. Ripper" "Sterling Hayden") ("Maj. T.J. \

"King\" Kong" "Slim Pickens") ("Dr. Strangelove" "Peter Sellers"))) (Oscars

("Best Actor (Nomin.)" "Best Adapted Screenplay (Nomin.)" "Best Director (N

omin.)" "Best Picture (Nomin.)")) (Sequel nil))

Thewhole out put app earsonone lon g linewit h minimal spaces, mak ingithardtoread.

Here’sthe sameout put manuallyfor matte d accordingtoS-expressionconvent ion s.Writing a

pretty-print erfor S-expressions islef t as a (challenging)exercis e;the dow nlo ad from gopl.io

includes a simpleversion.

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

SECTION 12.5. SETTING VARIABLES WITH REFLECT.V ALUE341

((Title "Dr. Strangelove")

(Subtitle "How I Learned to Stop Worrying and Love the Bomb")

(Year 1964)

(Actor (("Grp. Capt. Lionel Mandrake" "Peter Sellers")

("Pres. Merkin Muffley" "Peter Sellers")

("Gen. Buck Turgidson" "George C. Scott")

("Brig. Gen. Jack D. Ripper" "Sterling Hayden")

("Maj. T.J. \"King\" Kong" "Slim Pickens")

("Dr. Strangelove" "Peter Sellers")))

(Oscars ("Best Actor (Nomin.)"

"Best Adapted Screenplay (Nomin.)"

"Best Director (Nomin.)"

"Best Picture (Nomin.)"))

(Sequel nil))

Like the fmt.Print, json.Marshal,and Display func tions, sexpr.Marshal wi l l lo opforever

if cal le d with cyc lic dat a.

In Sec tion 12.6, we’ll sketch out the imp lementation of the cor respondingS-expressiondecod-

ingfunction, but beforeweget there , we’ll ﬁrs t ne e d to underst and howreﬂec tion can beused

to updateprogram variables.

Exercis e 12.3: Implementthe missingcas es of the encode func tion. Enco de boole ansas t and

nil,ﬂoating-p ointnumbers usingGo’snot ation,and complex numbers like1+2i as

#C(1.0 2.0).Int erfaces can beencoded asa pair ofatyp e name and a value,for ins tance

("[]int" (1 2 3)),but bewarethatthisnot ation isambiguous:the reflect.Type.String

method may retur n thesamestr ing for dif ferenttyp es.

Exercis e 12.4: Mo dif y encode to prett y-print the S-expression in the sty leshown above .

Exercis e 12.5: Ad apt encode to emitJSONins teadofS-expressions.Testyourencoder using

thestand ard decoder, json.Unmarshal.

Exercis e 12.6: Ad apt encode so that, as an opt imizat ion, itdoesnot encodeaﬁeldwhose

value isthe zerovalue ofits typ e.

Exercis e 12.7: Create a streamingAPI for the S-expressiondecoder,fol low ing the sty leof

json.Decoder (§4.5).

12.5. Setting Variables with reflect.Value

So far,reﬂec tion has only interpre ted values in our program in various ways. Thepoint ofthis

section, how ever, isto ch ange them.

Re call thatsom e Go expressions like x, x.f[1],and *p denot e var iables, but otherslike x+1

and f(2) do not.Avar iable isan ad dre ssabl e storagelocat ionthatcontainsavalue,and its

value may beupdated through thataddress.

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

342 CHAPTER 12. REFLECTION

Asimi lar dist inc tionapp lies to reflect.Values. Som e areaddressable; othersare not.

Consider the fol low ing declarat ions:

x:=2 // value type variable?

a:=reflect.ValueOf(2) // 2int no

b:=reflect.ValueOf(x) // 2int no

c:=reflect.ValueOf(&x) // &x *int no

d:=c.Elem() // 2int yes (x)

Thevalue wit hin a is not addressable. Itismerelyacopyofthe int eger2.The sameistrueof

b.The value wit hin c is als o non-addressable, beingacopyofthe point ervalue &x.Infac t, no

reflect.Value returned by reflect.ValueOf(x) is addressable. But d,der ive d from c by

dereferencingthe point erwit hin it, referstoavar iable andisthu s addressable. Wecan use

this appro ach,cal ling reflect.ValueOf(&x).Elem(),toobt ain an addressable Value forany

var iable x.

We can askareflect.Value whet her itisaddressablethrough its CanAddr method:

fmt.Println(a.CanAddr()) // "false"

fmt.Println(b.CanAddr()) // "false"

fmt.Println(c.CanAddr()) // "false"

fmt.Println(d.CanAddr()) // "true"

We obt ain an addressable reflect.Value whenever weindirec t thro ugh a point er, evenifwe

st arted fro m anon-addressable Value.All the usu alrules for addressabi lit y have analogs for

reﬂec tion.For example, since the slice indexingexpression e[i] implicitlyfol lowsapoint er, it

is addressableevenifthe expression e is not.Byanalog y, reflect.ValueOf(e).Index(i)

refers to a var iable,and isthu s addressableevenif reflect.ValueOf(e) is not.

To recov erthe var iable fro m an addressable reflect.Value re quires three steps. First,wecal l

Addr(),whichretur nsaValue holdingapointertothe var iable.Next, wecal l Interface()

on this Value,whichretur nsan interface{} value cont ainingthe point er. Final ly, ifwe

know the typ e of the var iable,wecan use a typ e assertiontoret rie vethe contentsofthe int er-

face as an ordinar y pointer. Wecan thenupdatethe var iable through the point er:

x:=2

d:=reflect.ValueOf(&x).Elem() // drefers to the variable x

px := d.Addr().Interface().(*int) // px := &x

*px = 3 // x = 3

fmt.Println(x) // "3"

Or,wecan updatethe var iable refer red tobyanaddressable reflect.Value direc tly,wit hout

usingapoint er, bycal lingthe reflect.Value.Set method:

d.Set(reflect.ValueOf(4))

fmt.Println(x) // "4"

Thesameche cks for assig nabilit y that areordinar ily per for med bythe compi ler aredon e at

runtimebythe Set methods. Above , thevar iable andthe value bot h have typ e int,but ifthe

var iable had beenan int64,the program wou ldpanic, soit’scruci al to makesurethe value is

assig nable tothe typ e of the var iable:

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

SECTION 12.5. SETTING VARIABLES WITH REFLECT.V ALUE343

d.Set(reflect.ValueOf(int64(5))) // panic: int64 is not assignable to int

Andofcours e calling Set on a non-addressable reflect.Value panics too:

x:=2

b:=reflect.ValueOf(x)

b.Set(reflect.ValueOf(3)) // panic: Set using unaddressable value

Thereare var iants of Set sp eci alizedfor cer tain gro ups ofbasic typ es: SetInt, SetUint, Set-

String, SetFloat,and soon:

d:=reflect.ValueOf(&x).Elem()

d.SetInt(3)

fmt.Println(x) // "3"

In som e ways these met hodsare morefor giv ing . SetInt,for example, will succe e d so lon g as

thevar iable’styp e is som e kind ofsig ned int eger, orevenanamed typ e whos e underly ing typ e

is a sig ned int eger, and ifthe value istoo large itwill bequiet lytrunc ate d to ﬁt. But tre adcare-

fully: cal ling SetInt on a reflect.Value that referstoan interface{} var iable will panic,

even thoug h Set wouldsucce e d.

x:=1

rx := reflect.ValueOf(&x).Elem()

rx.SetInt(2) // OK, x = 2

rx.Set(reflect.ValueOf(3)) // OK, x = 3

rx.SetString("hello") // panic: string is not assignable to int

rx.Set(reflect.ValueOf("hello")) // panic: string is not assignable to int

var y interface{}

ry := reflect.ValueOf(&y).Elem()

ry.SetInt(2) // panic: SetInt called on interface Value

ry.Set(reflect.ValueOf(3)) // OK, y = int(3)

ry.SetString("hello") // panic: SetString called on interface Value

ry.Set(reflect.ValueOf("hello")) // OK, y = "hello"

Wh enweapp lie d Display to os.Stdout,wefound thatreﬂec tion can readthe values of

unexp orted str uct ﬁelds thatare inaccessibleaccordingtothe usu alrules ofthe langu age, like

the fd int ﬁeldofan os.File st ruc t on a Unix-li keplatfor m.How ever, reﬂec tion cannot

up datesuchvalues:

stdout := reflect.ValueOf(os.Stdout).Elem() // *os.Stdout, an os.File var

fmt.Println(stdout.Type()) // "os.File"

fd := stdout.FieldByName("fd")

fmt.Println(fd.Int()) // "1"

fd.SetInt(2) // panic: unexported field

An addressable reflect.Value re cords whether itwas obt ained bytraversinganunexp orted

st ruc t ﬁeldand,ifso, dis allowsmodiﬁcation.Con sequently, CanAddr is not usu allythe rig ht

ch eck to use beforesetting a var iable.The rel ate d method CanSet reportswhether a

reflect.Value is addressable an d sett able:

fmt.Println(fd.CanAddr(), fd.CanSet()) // "true false"

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

344 CHAPTER 12. REFLECTION

12.6. Example: Decoding S-Expressions

Fo r each Marshal func tionprovide d by the stand ard librar y’s encoding/... packages, there is

acor responding Unmarshal func tionthatdoesdecoding. For example, aswesaw in

Section4.5, given a byteslice cont ainingJSON-enco deddat a forour Movie type (§12.3), we

candecodeitlikethis:

data := []byte{/* ... */}

var movie Movie

err := json.Unmarshal(data, &movie)

The Unmarshal func tionusesreﬂec tion tomodif y theﬁelds ofthe exist ing movie var iable,

creating new maps, str ucts, andslices as deter mined bythe typ e Movie andthe content ofthe

incomingdat a.

Let’snow imp lementasimple Unmarshal func tionfor S-expressions,analogou s to the stan-

dard json.Unmarshal func tionusedabove , andthe inv ers e of our earlier sexpr.Marshal.

We mustcaution you thatarobustand general implementation requires subst ant ial lymore

co de than will comfortably ﬁt in thisexample, whichisalready lon g,sowehavetaken many

shortcuts. Wesup por t on lyalimite d su bsetofS-expressions and donot handleerror s grace-

fully. The codeisint ended toillustratereﬂec tion,not parsing.

Thelexer usesthe Scanner type fro m the text/scanner packagetobre akaninp utstream

into a sequence oftokenssuchascomments, identiﬁers, str ing lit erals, andnumer ic literals.

Thescanner’s Scan method advances the scanner andretur nsthe kindofthe next token,

whichhas typ e rune.Mosttokens, like '(',con sistofasinglerune, but the text/scanner

packagerepresentsthe kinds ofthe multi-charac ter tokens Ident, String,and Int using

smal l negat ive values oftyp e rune.Fol low ing a cal l to Scan that retur nsone ofthese kinds of

token, the scanner’s TokenText method retur nsthe text ofthe token.

Sinceatypicalparsermay need toins pec t thecur renttoken several times, but the Scan

method advances the scanner,wewrap the scanner in a helpertyp e called lexer that keeps

trackofthe token mostrecentlyretur ned by Scan.

gopl.io/ch12/sexpr

type lexer struct {

scan scanner.Scanner

token rune // the current token

}

func (lex *lexer) next() {lex.token = lex.scan.Scan() }

func (lex *lexer) text() string { return lex.scan.TokenText() }

func (lex *lexer) consume(want rune) {

if lex.token != want { // NOTE: Not an example of good error handling.

panic(fmt.Sprintf("got %q, want %q", lex.text(), want))

}

lex.next()

}

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

SECTION 12.6. EXAMPLE: DECODING S-EXPRESSIONS 345

No w let’stur n to the parser. Itcon sists oftwo princip alfunctions.The ﬁrs t of these, read,

re ads the S-expressionthatstartswit h thecur renttoken andupdates the var iable refer red to

by the addressable reflect.Value v.

func read(lex *lexer, v reflect.Value) {

switch lex.token {

case scanner.Ident:

// The only valid identifiers are

// "nil" and struct field names.

if lex.text() == "nil" {

v.Set(reflect.Zero(v.Type()))

lex.next()

return

}

case scanner.String:

s, _ := strconv.Unquote(lex.text()) // NOTE: ignoring errors

v.SetString(s)

lex.next()

return

case scanner.Int:

i, _ := strconv.Atoi(lex.text()) // NOTE: ignoring errors

v.SetInt(int64(i))

lex.next()

return

case '(':

lex.next()

readList(lex, v)

lex.next() // consume ')'

return

}

panic(fmt.Sprintf("unexpected token %q", lex.text()))

}

OurS-expressions use identiﬁersfor two distinc t purpos es, st ruc t ﬁeldnames andthe nil

value for a point er. The read func tiononlyhandles the lattercas e.Whenitencount ers the

scanner.Ident "nil",itsets v to the zerovalue ofits typ e usingthe reflect.Zero func tion.

Fo r anyother identiﬁer,itrep ortsanerror.The readList func tion, whichwe’ll see ina

moment,handles identiﬁersused as str uct ﬁeldnames.

A '(' tokenindic ates the start ofalist. Thesecon d func tion, readList,decodes a list int o a

var iable ofcomposite typ e—amap,str uct,slice,orarray—dep endingonwhatkindofGo

var iable we’re cur rentlypopu lat ing. Ineachcas e,the loopkeeps parsingitems unt i l it encoun-

ters the matchingclos e parent hesis, ')', as detec ted bythe endList func tion.

Theint erest ing par t is the rec ursion. Thesimplestcas e is anarray.Unt i l theclosing ')' is

seen, weuse Index to obt ain the var iable for eacharray elementand makearec ursivecal l to

read to popu lateit. Asinmanyother erro r cases, if the inp utdat a caus esthe decoder toindex

beyond the end ofthe array,the decoder panics. A simi lar approach isusedfor slices, except

we mustcre ate a new var iable for eachelement, popu lateit, thenapp end ittothe slice.

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

346 CHAPTER 12. REFLECTION

Theloops for str uctsand maps mustparse a (key value) su blist oneachiterat ion. For

st ruc ts, thekey isasymbolidentifyingthe ﬁeld. Analogou s to the cas e forarrays, weobt ain

theexist ing var iable for the str uct ﬁeldusing FieldByName andmakearec ursivecal l to

popu lateit. For maps, the key may beofany typ e,and analogou s to the cas e forslices, wecre-

ateanew var iable,rec ursivelypopu lateit, and ﬁnallyins ert the newkey/value pair into the

map.

func readList(lex *lexer, v reflect.Value) {

switch v.Kind() {

case reflect.Array: // (item ...)

for i := 0; !endList(lex); i++ {

read(lex, v.Index(i))

}

case reflect.Slice: // (item ...)

for !endList(lex) {

item := reflect.New(v.Type().Elem()).Elem()

read(lex, item)

v.Set(reflect.Append(v, item))

}

case reflect.Struct: // ((name value) ...)

for !endList(lex) {

lex.consume('(')

if lex.token != scanner.Ident {

panic(fmt.Sprintf("got token %q, want field name", lex.text()))

}

name := lex.text()

lex.next()

read(lex, v.FieldByName(name))

lex.consume(')')

}

case reflect.Map: // ((key value) ...)

v.Set(reflect.MakeMap(v.Type()))

for !endList(lex) {

lex.consume('(')

key := reflect.New(v.Type().Key()).Elem()

read(lex, key)

value := reflect.New(v.Type().Elem()).Elem()

read(lex, value)

v.SetMapIndex(key, value)

lex.consume(')')

}

default:

panic(fmt.Sprintf("cannot decode list into %v", v.Type()))

}

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

SECTION 12.6. EXAMPLE: DECODING S-EXPRESSIONS 347

func endList(lex *lexer) bool {

switch lex.token {

case scanner.EOF:

panic("end of file")

case ')':

return true

}

return false

}

Final ly, wewrap upthe parserinanexp orted function Unmarshal,shown below,thathides

some ofthe rough edges ofthe imp lementation.Error s encountere d during parsingresultina

panic, so Unmarshal us esadefer red cal l to recov erfro m thepanic (§5.10) andretur n an erro r

mess age ins tead.

// Unmarshal parses S-expression data and populates the variable

// whose address is in the non-nil pointer out.

func Unmarshal(data []byte, out interface{}) (err error) {

lex := &lexer{scan: scanner.Scanner{Mode: scanner.GoTokens}}

lex.scan.Init(bytes.NewReader(data))

lex.next() // get the first token

defer func() {

// NOTE: this is not an example of ideal error handling.

if x := recover(); x != nil {

err = fmt.Errorf("error at %s: %v", lex.scan.Position, x)

}

}()

read(lex, reflect.ValueOf(out).Elem())

return nil

}

Apro duc tion-quality imp lementation shouldnever panic for any inp utand shouldrep ort an

infor mat ive error for every mishap, perhaps wit h alinenumberoroffset. Non etheless, we

hope thisexampleconve yssom e ide a of what’shappeningunder the hood ofthe packages like

encoding/json,and howyou can use reﬂec tion topopu latedat a st ruc tures.

Exercis e 12.8: The sexpr.Unmarshal func tion, like json.Marshal,requires the complete

inputinabyteslice beforeitcan beg in de coding. Deﬁne a sexpr.Decoder type that, like

json.Decoder,allowsasequence ofvalues tobedecoded fro m an io.Reader.Change

sexpr.Unmarshal to use thisnew typ e.

Exercis e 12.9: Wr ite a token-b ased API for decodingS-expressions,fol low ing the sty leof

xml.Decoder (§7.14). You will need ﬁve typ es of tokens: Symbol, String, Int, StartList,

and EndList.

Exercis e 12.10: Extend sexpr.Unmarshal to handlethe boole ans, ﬂoating-p ointnumbers,

andint erfaces enco dedbyyoursolut ion toExercis e 12.3. (Hint: todecodeint erfaces, you will

ne e d amapping fro m thenameofeachsup por ted typ e to its reflect.Type.)

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

348 CHAPTER 12. REFLECTION

12.7. Accessing Struct Field Tags

In Sec tion 4.5 weusedstr uct ﬁel d tags to modif y theJSONencodingofGostr uct values. The

json ﬁeldtag lets uscho ose alt ernat ive ﬁeldnames andsup pressthe out put of emp tyﬁelds.In

this sec tion,we’ll see how toaccess ﬁeldtags usingreﬂec tion.

In a web ser ver,the ﬁrs t thingmostHTTPhandler functions doisext ractthe requestparame-

ters int o lo cal variables. We’ll deﬁne a utilit y func tion, params.Unpack,thatusesstr uct ﬁeld

tags tomakewriting HTTPhandlers(§7.7) moreconvenient.

First,we’ll showhow it’sused. The search func tionbelow isanHTTPhandler.Itdeﬁnesa

var iable cal le d data of ananony mou s st ruc t type whose ﬁelds cor respond tothe HTTP

re questparameters. Thestr uct’sﬁeldtags specif y theparameter names, whichare often short

andcrypt icsince space ispre cious inaURL. The Unpack func tionpopu lates the str uct fro m

therequestsothatthe parameterscan beaccessedconvenientlyand wit h an appropriatetyp e.

gopl.io/ch12/search

import "gopl.io/ch12/params"

// search implements the /search URL endpoint.

func search(resp http.ResponseWriter, req *http.Request) {

var data struct {

Labels []string `http:"l"`

MaxResults int `http:"max"`

Exact bool `http:"x"`

}

data.MaxResults = 10 // set default

if err := params.Unpack(req, &data); err != nil {

http.Error(resp, err.Error(), http.StatusBadRequest) // 400

return

}

// ...rest of handler...

fmt.Fprintf(resp, "Search: %+v\n", data)

}

The Unpack func tionbelow doesthree things. First,itcal ls req.ParseForm() to parse the

re quest. Thereafter, req.Form cont ainsall the parameters, regardlessofwhether the HTTP

clientusedthe GET orthe POSTrequestmet hod.

Next, Unpack builds a mappingfro m the effec tiv e name ofeach ﬁeldtothe var iable for that

ﬁeld. The effec tive namemay differfro m theactualnameifthe ﬁeldhas a tag . The Field

method of reflect.Type returnsareflect.StructField that provides infor mat ionabout

thetyp e of each ﬁeldsuchasits name, typ e,and opt ion altag . The Tag ﬁeldisa

reflect.StructTag,whichisastr ing typ e that provides a Get method toparse and ext ract

thesubst ringfor a par tic ularkey,suchas http:"..." in thiscas e.

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

SECTION 12.7. ACCESSING STRUCT FIELD TAGS 349

gopl.io/ch12/params

// Unpack populates the fields of the struct pointed to by ptr

// from the HTTP request parameters in req.

func Unpack(req *http.Request, ptr interface{}) error {

if err := req.ParseForm(); err != nil {

return err

}

// Build map of fields keyed by effective name.

fields := make(map[string]reflect.Value)

v:=reflect.ValueOf(ptr).Elem() // the struct variable

for i := 0; i < v.NumField(); i++ {

fieldInfo := v.Type().Field(i) // a reflect.StructField

tag := fieldInfo.Tag // a reflect.StructTag

name := tag.Get("http")

if name == "" {

name = strings.ToLower(fieldInfo.Name)

}

fields[name] = v.Field(i)

}

// Update struct field for each parameter in the request.

for name, values := range req.Form {

f:=fields[name]

if !f.IsValid() {

continue // ignore unrecognized HTTP parameters

}

for _, value := range values {

if f.Kind() == reflect.Slice {

elem := reflect.New(f.Type().Elem()).Elem()

if err := populate(elem, value); err != nil {

return fmt.Errorf("%s: %v", name, err)

}

f.Set(reflect.Append(f, elem))

}else {

if err := populate(f, value); err != nil {

return fmt.Errorf("%s: %v", name, err)

}

return nil

}

Final ly, Unpack it erates overthe name/value pairsofthe HTTPparametersand updates the

correspondingstr uct ﬁelds.Recal l that the sameparameter namemay appear morethan

on ce. Ifthishappens,and the ﬁeldisaslice,thenall the values ofthatparameter areacc umu-

late d into the slice.Other wis e,the ﬁeldisrep eatedlyoverwritt ensothatonlythe lastvalue

hasany effec t.

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

350 CHAPTER 12. REFLECTION

The populate func tiontakes careofsetting a singleﬁeld v (orasingleelementofaslice ﬁeld)

from a parameter value.For now,itsup por tsonlystr ings, signe d integers,and boole ans.

Su pportingother typ es is lef t as an exercise.

func populate(v reflect.Value, value string) error {

switch v.Kind() {

case reflect.String:

v.SetString(value)

case reflect.Int:

i, err := strconv.ParseInt(value, 10, 64)

if err != nil {

return err

}

v.SetInt(i)

case reflect.Bool:

b, err := strconv.ParseBool(value)

if err != nil {

return err

}

v.SetBool(b)

default:

return fmt.Errorf("unsupported kind %s", v.Type())

}

return nil

}

If weadd the server hand ler toaweb ser ver,thismig htbeatypic al session:

$gobuild gopl.io/ch12/search

$./search &

$./fetch 'http://localhost:12345/search'

Search: {Labels:[] MaxResults:10 Exact:false}

$./fetch 'http://localhost:12345/search?l=golang&l=programming'

Search: {Labels:[golang programming] MaxResults:10 Exact:false}

$./fetch 'http://localhost:12345/search?l=golang&l=programming&max=100'

Search: {Labels:[golang programming] MaxResults:100 Exact:false}

$./fetch 'http://localhost:12345/search?x=true&l=golang&l=programming'

Search: {Labels:[golang programming] MaxResults:10 Exact:true}

$./fetch 'http://localhost:12345/search?q=hello&x=123'

x: strconv.ParseBool: parsing "123": invalid syntax

$./fetch 'http://localhost:12345/search?q=hello&max=lots'

max: strconv.ParseInt: parsing "lots": invalid syntax

Exercis e 12.11: Wr ite the cor responding Pack func tion. Given a str uct value, Pack should

return a URL incorporat ingthe parameter values fro m thestr uct.

Exercis e 12.12: Extend the ﬁeldtag not ation toexpress parameter validity requirements. For

example, a str ing mig htneed tobeavalid emai l addressorcre dit-c ardnumber, and anint eger

mig htneed tobeavalid US ZIP code.Modif y Unpack to checkthese requirements.

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

SECTION 12.8. DISPLAYING THE METHODS OF A TYPE 351

Exercis e 12.13: Mo dif y theS-expressionencoder (§12.4) anddecoder (§12.6) sothatthe y

honorthe sexpr:"..." ﬁeldtag in a similarmanner to encoding/json (§4.5).

12.8. Displaying the Methods ofaType

Ourﬁnalexampleofreﬂec tion uses reflect.Type to print the typ e of anarbit rar y value and

enumerateits met hods:

gopl.io/ch12/methods

// Print prints the method set of the value x.

func Print(x interface{}) {

v:=reflect.ValueOf(x)

t:=v.Type()

fmt.Printf("type %s\n", t)

for i := 0; i < v.NumMethod(); i++ {

methType := v.Method(i).Type()

fmt.Printf("func (%s) %s%s\n", t, t.Method(i).Name,

strings.TrimPrefix(methType.String(), "func"))

}

Both reflect.Type and reflect.Value have a met hod cal le d Method.Each t.Method(i)

call retur nsanins tance of reflect.Method,astr uct typ e that descr ibesthe nameand typ e of

asinglemet hod.Each v.Method(i) call retur nsareflect.Value repres enting a met hod

value (§6.4), thatis, a met hod boundtoits receiver. Usingthe reflect.Value.Call method

(w hichwedon’thavespace toshowhere), it’spossibletocal l Valuesofkind Func li kethisone,

butthisprogram needsonlyits Type.

Here are the methodsbelon gingtotwo typ es, time.Duration and *strings.Replacer:

methods.Print(time.Hour)

// Output:

// type time.Duration

// func (time.Duration) Hours() float64

// func (time.Duration) Minutes() float64

// func (time.Duration) Nanoseconds() int64

// func (time.Duration) Seconds() float64

// func (time.Duration) String() string

methods.Print(new(strings.Replacer))

// Output:

// type *strings.Replacer

// func (*strings.Replacer) Replace(string) string

// func (*strings.Replacer) WriteString(io.Writer, string) (int, error)

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

352 CHAPTER 12. REFLECTION

12.9. A Word ofCaution

Thereisalot moretothe reﬂec tion API thanwehavespace toshow, but the pre cedingexam-

ples giveanide a of whatispossible. Reﬂec tion isapow erfuland expressivetool, but it should

be usedwit h care , forthree reasons.

The ﬁrs t re asonisthatreﬂec tion-b ased codecan befrag i le.For every mistake thatwou ld

caus e acompi ler torep ort a typ e er ror,there isacor respondingway tomisus e reﬂec tion,but

whereasthe compi ler rep ortsthe mistake atbui ld time,areﬂec tion error isrep orted dur ing

exec ution asapanic, possibly lon g af ter the program was writt enorevenlon g af ter ithas

st arted running.

If the readList func tion(§12.6), for example, shouldreadastr ing fro m theinp utwhi le

popu lat ingavar iable oftyp e int,the cal l to reflect.Value.SetString wi l l panic. Most

prog ramsthatuse reﬂec tion havesimi lar hazards,and con siderable careisrequired tokeep

trackofthe typ e,addressabi lit y,and settabi lit y of each reflect.Value.

Thebestway toavoid thisfrag i lity istoens ure thatthe use ofreﬂec tion isfullyenc apsulated

within yourpackageand,ifpossible, avoid reflect.Value in favor of speciﬁctyp es in your

package’sAPI,torestr ict inp uts tolegal values. Ifthisisnot possible, per for m addition al

dy namic che cks beforeeachrisky operat ion. Asanexamplefro m thestand ard librar y,when

fmt.Printf applies a verb to an inappro priateoperand, itdoesnot panic mysteriou sly but

pr ints an infor mat ive error message . Theprogram still has a bug , butitiseasier todiagnos e.

fmt.Printf("%d %s\n", "hello", 42) // "%!d(string=hello) %!s(int=42)"

Reﬂec tion als o re duces the safet y andacc uracyofaut omated refac tor ing and analysistools,

becaus e they can’tdeter mineorrelyontyp e infor mat ion.

Thesecon d re asontoavoid reﬂec tion isthatsince typ es serveasafor m of documentation and

theoperat ions ofreﬂec tion cannot besubjec t to statictyp e ch eck ing , he avi lyreﬂec tive codeis

of ten hardtounderst and.Always caref ullydocumentthe exp ected typ es andother invar iants

of functions thatacceptan interface{} or a reflect.Value.

Thethirdreasonisthatreﬂec tion-b ased functions may beone ortwo ordersofmag nitude

slow erthancodespeci alizedfor a par tic ulartyp e.Inatypic al prog ram, themaj ority offunc-

tion s arenot relevanttothe overal l perfor mance,soit’sﬁne touse reﬂec tion whenitmakes the

prog ram clearer. Testing isapar tic ularlygood ﬁtfor reﬂec tion since mosttests use small dat a

sets. But for functions onthe criticalpat h, reﬂec tion isbestavoide d.

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

Lo w-Level Pro gramming

Thedesig n of Goguarante esanumberofsafet y prop erties thatlimitthe ways in whichaGo

prog ram can ‘‘go wro ng.’’ During compi lat ion, typ e ch eck ing detec tsmostatt emp tstoapp l y

an operat iontoavalue thatisinappro priatefor itstyp e,for ins tance,subtrac tingone str ing

from another.Str ict rules for typ e conv ersions pre ventdirec t accesstothe int ernalsofbui lt-in

typeslikestr ings, maps, slices, andchannel s.

Fo r er ror s that cannot bedetec ted statically, suchasout-of-b ounds array accessesornil

pointerdereferences, dynamic che cks ensure thatthe program immediate lyter minates wit h

an infor mat ive error whene ver a forbidden operat ionocc urs. Aut omat ic memory manage-

ment (garb agecol lec tion)eliminates ‘‘us e af ter fre e’’ bugs, as wel l as mostmemor y le aks.

Many imp lementation det ails are inaccessibletoGoprograms. Thereisnoway todis cov er

thememor y layout of anaggregatetyp e li kea str uct,orthe machinecodefor a function, or

theidentity ofthe operat ingsystemthreadonwhichthe cur rentgoroutine isrunning.

In deed,the Gosch edu ler fre ely mov esgoroutinesfro m on e thre adtoanother.Apointeriden-

tiﬁes a variablewit houtrevealingthe var iable’snumer ic address. Addressesmay change asthe

garb agecol lec tor mov esvar iables; point ers are transp arent lyupdated.

Together,these featuresmakeGoprograms, especi ally fai lingones, morepre dic table andless

myster ious thanprogramsinC,the quintess ent ial low-le vel langu age. Byhidingthe under-

ly ing det ails,the y also makeGoprogramshig hly por table,since the langu agesemantics are

large lyindep endentofany par tic ularcompi ler,operat ingsystem, orCPU archit ecture. (No t

entire lyindep endent: som e det ails leakthrough,suchasthe wordsize ofthe pro cessor, the

order ofevaluation of cer tain expressions,and the set ofimp lementation restr ictions imp osed

by the compi ler.)

Occasionally, wemay cho ose tofor feitsom e of these helpf ulguarante estoachie vethe hig hest

possible per for mance,toint ero peratewit h librar ies wr itt eninother langu ages, ortoimp le-

ment a functionthatcannot beexpress ed in pureGo.

353

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

354 CHAPTER 13. LOW-LEVEL PROGRAMMING

In thischapt er, we’ll see how the unsafe packagelets usstepoutside the usu alrules, andhow

to use the cgo to oltocre ate Gobindings for C librar ies andoperat ingsystemcal ls.

Theappro ach esdes crib edinthischapt ershouldnot beusedfrivo lou sly.Wit houtcaref ul

attent ion todet ail, the y maycause the kinds ofunp redic table,ins crutable, non-lo cal failures

with whichCprogrammersare unhappi lyacquainte d. Us e of unsafe also voids Go’swar ranty

of compatibi lit y with future rele ases, since,whether intended orinadver tent, itiseasy to

dep endonuns peciﬁed imp lementation det ails thatmay change unexp ectedly.

The unsafe packageisrat her mag ical.Alt hough itapp earstobearegu lar packageand is

imported inthe usu alway,itisactuallyimp lemente d by the compi ler.Itprovides accesstoa

numb erofbui lt-in langu agefeaturesthatare not ordinar ily avai lable because the y exp ose

det ails ofGo’smemor y layout.Present ing these featuresasasep aratepackagemakes the rare

occasions onwhichthe y areneeded morecon spicuou s.Als o,som e enviro nmentsmay restr ict

theuse ofthe unsafe packagefor sec urity reasons.

Package unsafe is used extensive lywit hin low-le vel packages like runtime, os, syscall,and

net that interac t with theoperat ingsystem, but isalmostnever needed byordinar y prog rams.

13.1. unsafe.Sizeof, Alignof,and Offsetof

The unsafe.Sizeof func tionrep ortsthe size in bytes ofthe represent ation of itsoperand,

whichmay beanexpressionofany typ e; theexpressionisnot evaluate d. Acal l to Sizeof is a

cons tantexpressionoftyp e uintptr,sothe resultmay beusedasthe dimensionofanarray

type,ortocompute other cons tants.

import "unsafe"

fmt.Println(unsafe.Sizeof(float64(0))) // "8"

Sizeof reportsonlythe size ofthe ﬁxe d part ofeachdat a st ruc ture, likethe point erand lengt h

of a str ing , butnot indirec t partslikethe contentsofthe str ing . Typicalsizes for all non-

ag gregateGotyp es areshown below,thoug h theexac t sizes may varybytoolchain. For

portabilit y,we’ve given the sizes ofreference typ es (ortyp es cont ainingreferences) in ter msof

word s,where a wordis4bytes ona32-bit platfor m and8bytes ona64-bit platfor m.

Comp uters loadand store values fro m memory mostefﬁcientlywhenthose values are

prop erly alig ned.For example, the addressofavalue ofatwo-bytetyp e such as int16 should

be anevennumber, the addressofafour-byte value suchasarune shouldbeamultipleof

four,and the addressofaneig ht-bytevalue suchasafloat64, uint64,or64-bit point er

shouldbeamultipleofeig ht. Alig nmentrequirements ofhig her multiples areunu sual, even

forlargerdat a typessuchas complex128.

Fo r this reason, the size ofavalue of an aggregatetyp e (a str uct orarray) is at least the sum of

thesizes ofits ﬁelds orelements but may begre aterdue tothe presenceof ‘‘holes.’’ Ho les are

unus edspaces addedbythe compi ler toens ure thatthe fol low ing ﬁeldorelementispro perly

alig ned rel ative tothe start ofthe str uct orarray.

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

SECTION 13.1. UNSAFE.SIZEOF,ALIGNOF,AND OFFSETOF 355

Ty peSiz e

bool 1byte

int

, uint

, float

, complex

/8bytes (for example, float64 is 8 bytes)

int, uint, uintptr 1word

*T 1word

string 2words (data,len)

[]T 3words (data,len, cap)

map 1word

func 1word

chan 1word

interface 2words (type,value)

Thelangu agespeciﬁcation doesnot guarante e that the order in which ﬁelds are declare d is the

order in whichthe y arelaid out inmemor y,sointhe ory a compi ler isfre e to rearrange them,

although aswewrite this, non e do.Ifthe typ es of a str uct’sﬁelds are ofdif ferentsizes, itmay

be morespace-efﬁcienttodeclare the ﬁelds inanorder thatpacks themastig htlyaspossible.

Thethree str uctsbelow havethe same ﬁelds,but the ﬁrs t re quires upto50% morememor y

than the other two:

// 64-bit 32-bit

struct{ bool; float64; int16 } // 3 words 4words

struct{ float64; int16; bool } // 2 words 3words

struct{ bool; int16; float64 } // 2 words 3words

Thedet ails ofthe alig nmentalgor it hmare beyon d thescope ofthisbook, andit’scer tain lynot

worthwor rying about every str uct,but efﬁcientpacking may makefre quentlyallocated dat a

st ruc tures morecompact and therefore faster.

The unsafe.Alignof func tionrep ortsthe required alig nmentofits argument’styp e.Like

Sizeof,itmay beapp lie d to anexpressionofany typ e,and ityieldsacon stant.Typic ally,

boole an andnumer ic typesare alig ned totheir size (uptoamaximum of8bytes) andall other

typesare word-alig ned.

The unsafe.Offsetof func tion, whose operandmustbeaﬁeldselec tor x.f,computesthe

of fsetofﬁeld f re lat ive tothe start ofits enclosingstr uct x,accounting for holes, if any.

Figure13.1 shows a str uct var iable x andits memor y layout on typic al 32- and64-bit Go

implementation s.The gray reg ions are holes.

var x struct {

abool

bint16

c[]int

}

Thetable below shows the results ofapp l yingthe three unsafe func tions to x itself andtoeach

of itsthree ﬁelds:

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

356 CHAPTER 13. LOW-LEVEL PROGRAMMING

Figure 13.1. Ho les in a str uct.

Typical 32-bit platform:

Sizeof(x) =16 Alignof(x) =4

Sizeof(x.a) = 1 Alignof(x.a) = 1 Offsetof(x.a) = 0

Sizeof(x.b) = 2 Alignof(x.b) = 2 Offsetof(x.b) = 2

Sizeof(x.c) = 12 Alignof(x.c) = 4 Offsetof(x.c) = 4

Typical 64-bit platform:

Sizeof(x) = 32 Alignof(x) =8

Sizeof(x.a) = 1 Alignof(x.a) = 1 Offsetof(x.a) = 0

Sizeof(x.b) = 2 Alignof(x.b) = 2 Offsetof(x.b) = 2

Sizeof(x.c) = 24 Alignof(x.c) = 8 Offsetof(x.c) = 8

Despit e their names, these functions are not infac t unsafe , andthe y maybehelpf ulfor under-

st andingthe layoutofraw memor y in a program whenopt imizingfor space.

13.2. unsafe.Pointer

Most point ertyp es arewritt en *T,meaning ‘‘apoint ertoavar iable oftyp e T.’’ The

unsafe.Pointer type isaspeci al kind ofpoint erthatcan holdthe addressofany var iable.Of

course,wecan’tindirec t thro ugh an unsafe.Pointer using *p becaus e we don’tknowwhat

type thatexpressionshouldhave. Likeordinar y pointers, unsafe.Pointersare comparable

andmay becompare d with nil,whichisthe zerovalue ofthe typ e.

An ordinar y *T pointermay beconverted toan unsafe.Pointer,and an unsafe.Pointer

maybeconverted back to anordinar y pointer, not necessarily ofthe sametyp e *T.Bycon-

vertinga*float64 pointertoa*uint64,for ins tance,wecan insp ect the bit pattern ofaﬂoat-

ing-p ointvar iable:

package math

func Float64bits(f float64) uint64 { return *(*uint64)(unsafe.Pointer(&f)) }

fmt.Printf("%#016x\n", Float64bits(1.0)) // "0x3ff0000000000000"

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

SECTION 13.2. UNSAFE.POINTER 357

Through the resulting point er, wecan updatethe bit pattern too.Thisishar mlessfor a ﬂoat-

ing-p ointvar iable since anybit pattern islegal,but ingeneral, unsafe.Pointer conv ersions

let uswrite arbit rar y values tomemor y andthu s su bvert the typ e system.

An unsafe.Pointer mayals o be converted toauintptr that holds the point er’snumer ic

value,letting usper for m ar ithmeticonaddresses. (Recal l from Chapt er3 thatauintptr is an

unsig ned int egerwide enough torepresent anaddress.) Thisconversiontoo may beapp lie d

in reverse,but again, conv ertingfro m a uintptr to an unsafe.Pointer maysubvert the typ e

systemsince not all numbers are valid addresses.

Many unsafe.Pointer values arethu s intermediaries for convertingordinar y pointers to raw

numericaddressesand backagain. Theexamplebelow takes the addressofvar iable x,adds

theoffsetofits b ﬁeld, convertsthe resulting addressto *int16,and through thatpoint er

up dates x.b:

gopl.io/ch13/unsafeptr

var x struct {

abool

bint16

c[]int

}

// equivalent to pb := &x.b

pb := (*int16)(unsafe.Pointer(

uintptr(unsafe.Pointer(&x)) + unsafe.Offsetof(x.b)))

*pb = 42

fmt.Println(x.b) // "42"

Although the syntaxiscumbers ome—perh aps nobad thingsince these featuresshouldbe

us edsparing ly—do not betempt edtoint roduce temporar y var iables oftyp e uintptr to bre ak

thelines. Thiscodeisincor rec t:

// NOTE: subtly incorrect!

tmp := uintptr(unsafe.Pointer(&x)) + unsafe.Offsetof(x.b)

pb := (*int16)(unsafe.Pointer(tmp))

*pb = 42

Thereasonisver y su btle. Som e garb agecol lec tor s move var iables around inmemor y to

re duce frag mentation or bookkeeping . Garb agecol lec tor s of thiskindare known as moving

GCs.Whenavar iable ismov ed, all point ers thatholdthe addressofthe old location mustbe

up dated topoint tothe newone.Fro m thepersp ectiveofthe garb agecol lec tor,an

unsafe.Pointer is a point erand thu s itsvalue mustchange asthe var iable mov es, but a

uintptr is justa numbersoits value mustnot change . Theincor rec t co de ab ove hi d esa

pointer from the garb agecol lec tor inthe non-p ointervar iable tmp.Bythe timethe secon d

st atement exe cut es, the var iable x couldhavemov edand the numberin tmp wouldnolon g er

be the address &x.b.The thirdstatement clobb ers anarbit rar y memory locat ionwit h the

value 42.

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

358 CHAPTER 13. LOW-LEVEL PROGRAMMING

Thereare myr iadpat holog ical variation s on thistheme.After thisstatement has exec ute d:

pT := uintptr(unsafe.Pointer(new(T))) // NOTE: wrong!

thereare nopoint ers thatrefer tothe var iable create d by new,sothe garb agecol lec tor is

entitled torec ycleits storagewhenthisstatement completes, afterwhich pT cont ainsthe

addresswhere the var iable was but isnolon g er.

No cur rentGoimp lementation usesamov ing garb agecol lec tor (though fut ure imp lemen-

tation s mig ht), but thisisnoreasonfor compl acency : currentversions of Go do mov e some

var iables around inmemor y.Recal l from Sec tion 5.2 thatgoroutine stacksgrow as needed.

Wh enthishappens,all var iables onthe old stack may berelocated toanew,largerstack,sowe

cannot relyonthe numer ic value ofavar iable’saddressremainingunchange d thro ughoutits

lifet ime.

At the timeofwriting , thereislit tle cle ar guid ance onwhatGoprogrammersmay relyupon

af ter an unsafe.Pointer to uintptr conv ersion(seeGoissue7192), sowestron gly recom-

mend thatyou assume the bareminimum. Tre atall uintptr values as if the y cont ain the

for mer addressofavar iable,and minimize the numberofoperat ions bet weenconvertingan

unsafe.Pointer to a uintptr andusingthat uintptr.Inour ﬁrs t exampleabove , thethree

op erat ions—conv ersiontoauintptr,addition of the ﬁeldoffset, conv ersionback—al l

appeared wit hin a singleexpression.

Wh encal lingalibrar y func tionthatretur nsauintptr,such as those below fro m the reflect

package, the resultshouldbeimmediate lyconverted toan unsafe.Pointer to ens ure thatit

cont inues topoint tothe samevar iable.

package reflect

func (Value) Pointer() uintptr

func (Value) UnsafeAddr() uintptr

func (Value) InterfaceData() [2]uintptr // (index 1)

13.3. Example: Deep Equivalence

The DeepEqual func tionfro m the reflect packagerep ortswhether two values are ‘‘de eply’’

equal. DeepEqual comp aresbasic values as if bythe bui lt-in == op erator ; forcomposite val-

ues, ittraversesthemrec ursively, comparing cor respondingelements. Because itwor ksfor

anypair ofvalues, evenonesthatare not comparable wit h ==,itﬁnd s widespreaduse intests.

Thefol low ing testuses DeepEqual to compare two []string values:

func TestSplit(t *testing.T) {

got := strings.Split("a:b:c", ":")

want := []string{"a", "b", "c"};

if !reflect.DeepEqual(got, want) { /* ... */ }

}

Although DeepEqual is convenient, its dist inc tions can seemarbit rar y.For example, itdoesn’t

consider a nilmap equ altoanon-ni l empt y map, nor a nil slice equ altoanon-ni l empt y on e:

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

SECTION 13.3. EXAMPLE: DEEP EQUIVALENCE 359

var a, b []string = nil, []string{}

fmt.Println(reflect.DeepEqual(a, b)) // "false"

var c, d map[string]int = nil, make(map[string]int)

fmt.Println(reflect.DeepEqual(c, d)) // "false"

In thissec tion we’ll deﬁne a function Equal that comp aresarbit rar y values. Like DeepEqual,

it comparesslices andmaps bas edontheir elements, but unlike DeepEqual,itcon sidersanil

slice (ormap) equ altoanon-ni l empt y on e.The basic rec ursionoverthe arguments can be

done wit h reﬂec tion,usingasimi lar approach to the Display prog ram we saw in Sec tion 12.3.

As usu al, wedeﬁne anunexp orted function, equal,for the rec ursion. Don’twor ryabout the

seen parameter justyet. For eachpair ofvalues x and y to becompare d, equal ch ecksthat

both (or neither) arevalid andche cks thatthe y have the sametyp e.The resultofthe function

is deﬁne d as a set ofswitch cas es that comp are two values ofthe sametyp e.For reasons of

sp ace,we’ve omitt edseveral cas es since the pattern shouldbefami liar bynow.

gopl.io/ch13/equal

func equal(x, y reflect.Value, seen map[comparison]bool) bool {

if !x.IsValid() || !y.IsValid() {

return x.IsValid() == y.IsValid()

}

if x.Type() != y.Type() {

return false

}

// ...cycle check omitted (shown later)...

switch x.Kind() {

case reflect.Bool:

return x.Bool() == y.Bool()

case reflect.String:

return x.String() == y.String()

// ...numeric cases omitted for brevity...

case reflect.Chan, reflect.UnsafePointer, reflect.Func:

return x.Pointer() == y.Pointer()

case reflect.Ptr, reflect.Interface:

return equal(x.Elem(), y.Elem(), seen)

case reflect.Array, reflect.Slice:

if x.Len() != y.Len() {

return false

}

for i := 0; i < x.Len(); i++ {

if !equal(x.Index(i), y.Index(i), seen) {

return false

}

return true

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

360 CHAPTER 13. LOW-LEVEL PROGRAMMING

// ...struct and map cases omitted for brevity...

}

panic("unreachable")

}

As usu al, wedon’texp osethe use ofreﬂec tion inthe API,sothe exp orted function Equal

mu stcal l reflect.ValueOf on itsarguments:

// Equal reports whether x and y are deeply equal.

func Equal(x, y interface{}) bool {

seen := make(map[comparison]bool)

return equal(reflect.ValueOf(x), reflect.ValueOf(y), seen)

}

type comparison struct {

x, y unsafe.Pointer

treflect.Type

}

To ens ure thatthe algor it hmter minates evenfor cyc lic dat a st ruc tures, itmustrecordwhich

pairsofvar iables ithas already compare d andavoid comp aring themasecon d time. Equal

al locatesaset of comparison st ruc ts, each holdingthe addressoftwo var iables (represent edas

unsafe.Pointer values) andthe typ e of the comparison. Weneed torecordthe typ e in addi-

tion tothe addressesbecause dif ferentvar iables can havethe sameaddress. For example, if x

and y arebot h ar rays, x and x[0] have the sameaddress, as do y and y[0],and itisimp ortant

to distinguish whether wehavecompare d x and y or x[0] and y[0].

Once equal hasest ablishe d that its arguments havethe sametyp e,and beforeitexe cut esthe

sw itch,itche cks whether itiscomparing two var iables ithas already seenand,ifso, ter minates

therec ursion.

// cycle check

if x.CanAddr() && y.CanAddr() {

xptr := unsafe.Pointer(x.UnsafeAddr())

yptr := unsafe.Pointer(y.UnsafeAddr())

if xptr == yptr {

return true // identical references

}

c:=comparison{xptr, yptr, x.Type()}

if seen[c] {

return true // already seen

}

seen[c] = true

}

Here’sour Equal func tion in action:

fmt.Println(Equal([]int{1, 2, 3}, []int{1, 2, 3})) // "true"

fmt.Println(Equal([]string{"foo"}, []string{"bar"})) // "false"

fmt.Println(Equal([]string(nil), []string{})) // "true"

fmt.Println(Equal(map[string]int(nil), map[string]int{})) // "true"

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

SECTION 13.4. CALLING C CODE WITH CGO 361

It evenwor ksoncyc lic inputs similartothe one thatcausedthe Display func tionfro m

Section12.3 toget stuck inaloop:

// Circular linked lists a -> b -> a and c -> c.

type link struct {

value string

tail *link

}

a, b, c := &link{value: "a"}, &link{value: "b"}, &link{value: "c"}

a.tail, b.tail, c.tail = b, a, c

fmt.Println(Equal(a, a)) // "true"

fmt.Println(Equal(b, b)) // "true"

fmt.Println(Equal(c, c)) // "true"

fmt.Println(Equal(a, b)) // "false"

fmt.Println(Equal(a, c)) // "false"

Exercis e 13.1: Deﬁne a deepcomparisonfunctionthatcon sidersnumbers (of any typ e) equal

if the y dif fer bylessthanone par t in a billion.

Exercis e 13.2: Wr ite a functionthatrep ortswhether its argumentisacyc lic dat a st ruc ture.

13.4. Calling CCode with cgo

AGoprogram might need touse a hardwaredriverimp lemente d in C, quer y an embedde d

database imp lemente d in C++, oruse some linearalgebra routinesimp lemente d in For tran. C

haslon g been the lingu a franca ofprogramming, somanypackages intended for widespread

us e exp ort a C-comp atibleAPI,regardlessofthe langu ageoftheir implementation.

In thissec tion,we’ll bui ld asimpledat a comp ressionprogram thatuses cgo,atoolthatcre ates

Go bindings for C functions.Suchtools are cal le d foreign-f unc tionint erfaces (FFIs), and cgo is

notthe onlyone for Goprograms. SWIG(swig.org)isanother ; it prov ides morecomplex

fe aturesfor int egrat ingwit h C++ class es, but wewon’tshowithere.

The compress/... su btree ofthe stand ard librar y prov ides comp ressors and decompressors

forpopu lar comp ressionalgor it hms,includingLZW (usedbythe Unix compress command)

andDEFL ATE (usedbythe GNU gzip command). TheAPIsofthese packages varyslig htly

in detai ls, but the y al l prov ide a wrapperfor an io.Writer that comp ressesthe dat a wr itt ento

it,and a wrapperfor an io.Reader that decomp ressesthe dat a re adfro m it.For example:

package gzip // compress/gzip

func NewWriter(w io.Writer) io.WriteCloser

func NewReader(r io.Reader) (io.ReadCloser, error)

Thebzip2 algor it hm, whichisbas edonthe elegantBur rows-Whe eler transfor m,runsslower

than gzipbut yieldssig niﬁcant lybettercompression. The compress/bzip2 packageprovides

adecompressorfor bzip2, but atthe moment the packageprovides nocompressor. Bui lding

on e from scratch isasubst ant ial under tak ing, but there isawel l-do cumente d andhig h-per-

formance open-s ource C implementation,the libbzip2 packagefro m bzip.org.

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

362 CHAPTER 13. LOW-LEVEL PROGRAMMING

If the C librar y were small,wewou ldjustpor t it topureGo, and ifits per for mance werenot

cr iticalfor our pur pos es, we wouldbebetteroff inv oking a C program as a helpersubprocess

usingthe os/exec package. It’swhenyou need touse a complex, per for mance-cr iticallibrar y

with a nar row C API thatitmay makesense towrap itusing cgo.For the restofthischapt er,

we’ll wor k thro ugh anexample.

Fr omthe libbzip2 Cpackage, weneed the bz_stream st ruc t type,whichholds the inp utand

output buf fers, andthree C functions: BZ2_bzCompressInit,whichallocates the stream’s

buffers; BZ2_bzCompress,whichcompressesdat a from the inp utbuf fer tothe out put buf fer ;

and BZ2_bzCompressEnd,whichreleasesthe buf fers. (Don’twor ryabout the mech anics ofthe

libbzip2 package; the pur pos e of thisexampleistoshowhow the par ts ﬁttoget her.)

We’ll cal l the BZ2_bzCompressInit and BZ2_bzCompressEnd Cfunctions direc tly fro m Go,

butfor BZ2_bzCompress,we’ll deﬁne a wrapperfunctioninC,toshowhow it’sdon e.The C

source ﬁle below lives alongside the Gocode in our package:

gopl.io/ch13/bzip

/* This file is gopl.io/ch13/bzip/bzip2.c, */

/* a simple wrapper for libbzip2 suitable for cgo. */

#include <bzlib.h>

int bz2compress(bz_stream *s, int action,

char *in, unsigned *inlen, char *out, unsigned *outlen) {

s->next_in = in;

s->avail_in = *inlen;

s->next_out = out;

s->avail_out = *outlen;

int r = BZ2_bzCompress(s, action);

*inlen -= s->avail_in;

*outlen -= s->avail_out;

return r;

}

No w let’stur n to the Gocode, the ﬁrs t part ofwhichisshown below.The import "C" de clara-

tion isspeci al.There isnopackage C,but thisimp ort causes go build to preprocessthe ﬁle

usingthe cgo to olbeforethe Gocompi ler seesit.

// Package bzip provides a writer that uses bzip2 compression (bzip.org).

package bzip

#cgo CFLAGS: -I/usr/include

#cgo LDFLAGS: -L/usr/lib -lbz2

#include <bzlib.h>

int bz2compress(bz_stream *s, int action,

char *in, unsigned *inlen, char *out, unsigned *outlen);

import "C"

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

SECTION 13.4. CALLING C CODE WITH CGO 363

import (

"io"

"unsafe"

)

type writer struct {

wio.Writer // underlying output stream

stream *C.bz_stream

outbuf [64 * 1024]byte

}

// NewWriter returns a writer for bzip2-compressed streams.

func NewWriter(out io.Writer) io.WriteCloser {

const (

blockSize =9

verbosity = 0

workFactor = 30

)

w:=&writer{w: out, stream: new(C.bz_stream)}

C.BZ2_bzCompressInit(w.stream, blockSize, verbosity, workFactor)

return w

}

During preprocessing, cgo generates a temporar y packagethatcontainsGodeclarat ions cor-

resp ondingtoall the C functions and typ es us edbythe ﬁle,suchas C.bz_stream and

C.BZ2_bzCompressInit.The cgo to oldis cov ers these typ es by inv oking the C compi ler in a

sp eci al way onthe contentsofthe comment thatpre cedes the imp ort declarat ion.

Thecomment may also contain #cgo direc tivesthatspecif y ext ra opt ion s to the C toolchain.

The CFLAGS and LDFLAGS values cont ribut e ext ra arguments tothe compi ler andlin ker com-

mand s so thatthe y canlocatethe bzlib.h he ader ﬁle andthe libbz2.a archivelibrar y.The

exampleassumesthatthese are ins tal le d bene ath /usr on yoursystem. You may need toalt er

or deletethese ﬂags for yourins tal lat ion.

NewWriter makesacal l to the C function BZ2_bzCompressInit to initialize the buf fersfor

thestream. The writer type includes anot her buf fer thatwill beusedtodrain the decom-

pressor’sout put buf fer.

The Write method,shown below,feedsthe uncompressed data to the compressor, cal lingthe

func tion bz2compress in a loop unt i l al l thedat a hasbeencon sumed.Obs erve thatthe Go

prog ram mayaccessC typ es li ke bz_stream, char,and uint,Cfunctions like bz2compress,

andevenobj e ct-li keCprepro cessormacros suchas BZ_RUN,all through the C.x notation.The

C.uint type isdistinc t from Go’s uint type,even if bot h have the samewidth.

func (w *writer) Write(data []byte) (int, error) {

if w.stream == nil {

panic("closed")

}

var total int // uncompressed bytes written

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

364 CHAPTER 13. LOW-LEVEL PROGRAMMING

for len(data) > 0 {

inlen, outlen := C.uint(len(data)), C.uint(cap(w.outbuf))

C.bz2compress(w.stream, C.BZ_RUN,

(*C.char)(unsafe.Pointer(&data[0])), &inlen,

(*C.char)(unsafe.Pointer(&w.outbuf)), &outlen)

total += int(inlen)

data = data[inlen:]

if _, err := w.w.Write(w.outbuf[:outlen]); err != nil {

return total, err

}

return total, nil

}

Each iterat ionofthe looppasses bz2compress theaddressand lengt h of the remaining

portionof data,and the addressand cap acity of w.outbuf.The two lengt h var iables are

pass edbytheir addresses, not their values, sothatthe C functioncan updatethemtoindic ate

howmuchuncompresseddat a was cons ume d andhow muchcompresseddat a was pro duce d.

Each chun k of compresseddat a is thenwritt entothe underly ing io.Writer.

The Close method has a similarstr uctureto Write,usingalooptoﬂushout any remaining

comp resseddat a from the stream’sout put buf fer.

// Close flushes the compressed data and closes the stream.

// It does not close the underlying io.Writer.

func (w *writer) Close() error {

if w.stream == nil {

panic("closed")

}

defer func() {

C.BZ2_bzCompressEnd(w.stream)

w.stream = nil

}()

for {

inlen, outlen := C.uint(0), C.uint(cap(w.outbuf))

r:=C.bz2compress(w.stream, C.BZ_FINISH, nil, &inlen,

(*C.char)(unsafe.Pointer(&w.outbuf)), &outlen)

if _, err := w.w.Write(w.outbuf[:outlen]); err != nil {

return err

}

if r == C.BZ_STREAM_END {

return nil

}

Up oncomplet ion, Close calls C.BZ2_bzCompressEnd to release the streambuf fers, using

defer to ens ure thatthishappens onall retur n paths. Atthispoint the w.stream pointeris

no lon g ersafetodereference.Tobedefensive , we set itto nil,and add explicitnil che cks to

each met hod,sothatthe program panics if the usermistakenlycal lsamet hod after Close.

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

SECTION 13.4. CALLING C CODE WITH CGO 365

No t on lyis writer notcon cur rency-s afe, but con cur rentcal lsto Close and Write couldcause

theprogram tocrash in C code. FixingthisisExercis e 13.3.

Theprogram below, bzipper,isabzip2 comp ressorcommand thatusesour new package. It

behaveslikethe bzip2 commandpresent onmanyUnix systems.

gopl.io/ch13/bzipper

// Bzipper reads input, bzip2-compresses it, and writes it out.

package main

import (

"io"

"log"

"os"

"gopl.io/ch13/bzip"

)

func main() {

=bzip.NewWriter(os.Stdout)

if _, err := io.Copy(w, os.Stdin); err != nil {

log.Fatalf("bzipper: %v\n", err)

}

if err := w.Close(); err != nil {

log.Fatalf("bzipper: close: %v\n", err)

}

In the sessionbelow,weuse bzipper to compress /usr/share/dict/words,the systemdic-

tion ary,fro m 938,848 bytes to335,405 bytes—ab out a thirdofits originalsize—then uncom-

pressitwit h thesystem bunzip2 command. The SHA256 hashisthe samebeforeand after,

giving usconﬁdence thatthe compressoriswor kingcor rec tly.(If you don’thave sha256sum

on yoursystem, use yoursolut ion toExercis e 4.2.)

$gobuild gopl.io/ch13/bzipper

$wc-c</usr/share/dict/words

938848

$sha256sum < /usr/share/dict/words

126a4ef38493313edc50b86f90dfdaf7c59ec6c948451eac228f2f3a8ab1a6ed -

$./bzipper < /usr/share/dict/words | wc -c

335405

$./bzipper < /usr/share/dict/words | bunzip2 | sha256sum

126a4ef38493313edc50b86f90dfdaf7c59ec6c948451eac228f2f3a8ab1a6ed -

We’ve demon strated lin kingaClibrar y into a Goprogram. Going inthe other direc tion,it’s

also possibletocompi leaGoprogram as a staticarc hivethatcan belin ked int o aCprogram

or asashare d librar y that can bedynamic ally loade d by a C program. We’ve onlyscratch ed

thesur face of cgo here , andthere ismuchmoretosay aboutmemor y management,point ers,

callbacks, signalhandling, str ings, errno,ﬁnalizers, andthe rel ation shipbet weengoroutines

andoperat ingsystemthreads,muchofitver y su btle. Inpar tic ular, the rules for cor rec tly

passingpoint ers fro m Go toCorvice versa are complex, for reasons simi lar tothose we

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

366 CHAPTER 13. LOW-LEVEL PROGRAMMING

discussedinSec tion 13.2, andnot yet author it ative lyspeciﬁed.For fur therreading, start wit h

https://golang.org/cmd/cgo.

Exercis e 13.3: Us e sync.Mutex to make bzip2.writer safe forcon cur rentuse bymultiple

goro utines.

Exercis e 13.4: DependingonClibrar ies hasits drawbacks. Provide an alternat ive pure-G o

implementation of bzip.NewWriter that usesthe os/exec packagetorun /bin/bzip2 as a

su bprocess.

13.5. Another WordofCaution

We ended the pre vious chapt erwit h awar ningabout the dow nsides ofthe reﬂec tion int erface.

That warningapp lies wit h even moreforce tothe unsafe packagedes crib ed in thischapt er.

High-le vel langu ages insu lateprogramsand programmersnot onlyfro m thearc ane speciﬁcs

of indiv idu alcomputerins tructionsets, but fro m dep endence onirrelevancies like where in

memory a var iable lives, how big a dat a type is, the det ails ofstr ucturelayout, andahostof

ot her implementation det ails.Because ofthatins ulating layer,it’spossibletowrite programs

that aresafeand robustand thatwill run onany operat ingsystemwit houtchange .

The unsafe packagelets programmersreach through the ins ulation touse some cruci al but

ot her wis e inaccessiblefeature , or perhaps toachie vehig her per for mance.The costisusu ally

to por tabilit y andsafet y,soone uses unsafe at one’sper il. Our advice onhow and whento

us e unsafe paral lels Knu th’scommentsonpremature opt imizat ion, whichwequote d in

Section11.5. Mostprogrammerswill never need touse unsafe at all. Never theless, there will

occasionallybesit uat ions where som e cr iticalpie ce of codecan bebestwritt enusing unsafe.

If caref ulstudy and measurementindic ates that unsafe re allyisthe bestappro ach,restr ict it

to assmall a reg ion as possible, sothatmostofthe program isobliv iou s to its use.

Fo r now, put the lasttwo chapt ers inthe backofyourmind. Write som e su bst ant ial Go

prog rams. Avoid reflect and unsafe;com e back to these chapt ers only if you must.

Me anw hile, happy Goprogramming. Wehop e youenj oywriting Go as much as we do.

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

Index

!,negat ionoperator 63

%,remainder operator 52, 166

&&,short-circuit AND operator 63

&,address-ofoperator 24, 32, 94,

158, 167

&,imp licit158, 167

&^,AND-NOT operator 53

&^,bit-cle ar op erator 53

' quot e ch arac ter 56

*,indirec tion operator 24, 32

++,increment statement 5,37, 94

+,str ing con catenat ionoperator 5,

+,unary operator 53

+=, -=,etc., assig nmentoperator 5

-,unary operator 53

--,decrement statement 5,37

... argument139, 142

... ar ray lengt h 82

... parameter 91, 142, 143, 172

... path 292, 299

/*...*/ comment5,25

// comment5,25

:= short var iable decl arat ion5,31,

<<,lef t shif t op erator 54

==,comparisonoperator 40, 63

>>,rig htshif t op erator 54

^,bit w ise complementoperator 53

^,exc lusiveORoperator 53

_,blank identiﬁer 7, 38, 95, 120, 126,

287

` backquot e ch arac ter 66

| in templ ate 113

|,bit w ise ORoperator 166, 167

||,short-circuit ORoperator 63

Abst rac t Sy ntaxNot ation One

(ASN.1) 107

abstrac t type 24, 171

abstrac tion, premature 216, 316, 317

ad hoc poly mor phism 211

addressoflocal variable32, 36

addressofstr uct lit eral 103

addressableexpression159, 341

addressablevalue 32

address-ofoperator & 24, 32, 94,

158, 167

ag gregatetyp e 81, 99

Alef programminglangu agexiii

algor it hm

breadt h-ﬁrs t search 139, 239

depth-ﬁrs t search 136

Fibonacci 37, 218

GCD 37

insertionsor t 101

Liss ajous 15

slice rot ation 86

topolog ical sor t 136

ali asing, point er33

alig nment354

al location

he ap36

memory 36, 71, 89, 169, 209, 322

st ack 36

anch orelement, HTML 122

AND operator &&,short-circuit 63

AND-NOT operator &^ 53

animat ion, GIF 13

anonymou s

func tion22, 135, 236

func tion, defer 146

func tion, rec ursive137

st ruc t ﬁeld104, 105, 106, 162

API

encoding 213, 340

er ror 127, 152

package284, 296, 311, 333, 352

runt ime 324

SQL211

systemcal l 196

template 115

token-b ased decoder 213, 215,

347

APLprogramminglangu agexiii

append built-in function88, 90, 91

appendInt example88

argument

... 139, 142

command-line4,18, 33, 43, 179,

180, 290, 313

func tion119

pointer33, 83

slice 86

ar ithmeticexpressionevaluator197

ar ray

comp arison83

lengt h, ... 82

literal 82, 84

type 81

underly ing 84, 88, 91, 187

zerovalue 82

ASCII 56, 64, 66, 67, 305

ASN.1 (Abstrac t Sy ntaxNot ation

One) 107

assemb l y line, cake234

assertion

func tion316

interface typ e 208, 210

367

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

368 INDEX

test 306

type 205, 211

assig nabilit y 38, 175

assig nabilit y,int erface 175

assig nment

implicit38

mu ltiple-value 37

op erator +=, -=,etc.5

op erator s 36, 52

st atement 5, 7, 36, 52, 94, 173

tuple31, 37

asso ciativity,operator 52

atomic operat ion264

attack,HTMLinj e ction 115

attack,SQL inj e ction 211

autoescape example117

back-do or, package315

back-off,exp onent ial 130

backquot e ch arac ter, ` 66

bank examplepackage258, 261, 263

bare return 126

basename example72

behavior,undeﬁne d 260

Benchmark func tion302, 321

bidirec tion altounidirec tion al

ch annel conversion231

binary

op erator s,table of52

semaph ore 262

tree 102

bit vec tor 165

bit-cle ar op erator &^ 53

bit-s et data typ e 77

bit w ise

comp lementoperator ^ 53

op erator s,table of53

OR operator | 166, 167

bl ack-b oxtest310

bl ank identiﬁer _ 7, 38, 95, 120, 126,

287

bl ank imp ort 287

block

ﬁle 46

lexic al 46, 120, 135, 141, 212

lo cal 46

package46

univers e 46

blocking proﬁle 324

Blog , Go xvi,326

boiling example29

bool type 63

boole an

cons tant, false 63

cons tant, true 63

zerovalue 30

breadthFirst func tion139

breadt h-ﬁrs t search algor it hm139,

239

break st atement 24, 46

break st atement,lab ele d 249

br ittletest317

broadc ast251, 254, 276

Brooks, Fre d xiv

btoi func tion64

buffered channel 226, 231

bufio package9

bufio.NewReader func tion98

bufio.NewScanner func tion9

(*bufio.Reader).ReadRune

method 98

bufio.Scanner type 9

(*bufio.Scanner).Err method 97

(*bufio.Scanner).Scan method 9

(*bufio.Scanner).Split method

bufio.ScanWords func tion99

+build comments 296

buildcon straints 296

buildtags 296

buildingpackages 293

built-in function

append 88, 90, 91

cap 84, 232

close 226, 228, 251

complex 61

copy 89

delete 94

imag 61

len 4, 54, 64, 65, 81, 84, 233

make 9, 18, 88, 94, 225

new 34

panic 148, 149

real 61

recover 152

built-in interface, error 196

built-in typ e, error 11, 128, 149,

196

byte slice tostr ing conversion73

byte type 52

ByteCounter example173

bytes package71, 73

bytes.Buffer type 74, 169, 172, 185

(*bytes.Buffer).Grow method

169

(*bytes.Buffer).WriteByte

method 74

(*bytes.Buffer).WriteRune

method 74

(*bytes.Buffer).WriteString

method 74

bytes.Equal func tion86

bzip Ccode362

bzip examplepackage363

bzipper example365

C++ programminglangu agexiv,xv,

361

Cprogramminglangu agexii,xv, 1,

6, 52, 260, 361

cach e, conc urrentnon-blo cking 272

cach e, non-blo cking 275

cake ass emb l y line234

call

by reference 83

by value 83, 120, 158

interface met hod 182

ok value fro m func tion128

callingCfro m Go 361

camel cas e 28

cancel lat ion251, 252

cancel lat ionofHTTPrequest253

cap built-in function84, 232

capacity,channel 226, 232, 233

capacity,slice 88, 89

capt uring iterat ionvar iable 140

capt uring loopvar iable 141, 236,

240

case in typ e switch 212

case, select 245

Celsius type 39

CelsiusFlag func tion181

cf example43

cgo to ol361, 362

<-ch,channel receive 18, 225, 232

ch<-,channel send18, 225, 232

ch aining, met hod 114

chan type 225

ch annel

buffered 226, 231

capacity 226, 232, 233

close 228, 251

closing a 225

communic ation225, 245

comp arison225

conv ersion, bidirec tion alto

unidirec tion al231

draining a 229, 252

make 18, 225

ni l 246, 249

polling246

range ov er229

re ceive <-ch 18, 225, 232

re ceive , non-blo cking 246

re ceive , ok value fro m 229

send ch<- 18, 225, 232

sy nchro nou s 226

type 18

type <-chan T,receive-only230

type chan<- T,send-only230

type,unidirec tion al230, 231

unbuffered 226

zerovalue 225, 246

ch arac ter conv ersion71

ch arac ter test71

charcount example98

chat example254

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

INDEX 369

ch atser ver 253

CheckQuota func tion312, 313

client, emai l 312

client, SMTP 312

clock example220, 222

clockser ver,con cur rent219

close built-in function226, 228,

251

close,channel 228, 251

clos er goro utine 238, 250

closingachannel 225

closure, lexic al 136

cmplx.Sqrt func tion61

co de

format 3, 6, 9, 48

point, Unico de 67

produc tion301

ColoredPoint example161

comma example73

command, test ing a 308

command-lineargument4,18, 33,

43, 179, 180, 290, 313

comment

/*...*/ 5, 25

// 5, 25

do c 42, 296

// Output 326

comments, +build 296

communic atingsequential pro cesses

(CSP) xiii,217

communic ation, channel 225, 245

comp arabilit y 9, 38, 40, 53, 86, 93,

97, 104

comp arison

ar ray 83

ch annel 225

func tion133

interface 184

map96

op erator == 40, 63

op erator s 40, 93

op erator s,table of53

slice 87

st ring65

st ruc t 104

comp ilation,sep arate284

comp lementoperator ^,bit w ise 53

complex built-in function61

complex type 61

comp osite lit eral 14

comp osite typ e xv,14, 81

comp osition,paral lel224

comp osition,typ e xv,107, 162, 189

compress/bzip2 package361

comp ression361

conceptu alint egr ity xiv

concrete typ e 24, 171, 211, 214

conc urrency 17, 217, 257

excessive 241, 242

safe 275

safety 256, 257, 272, 365

with share d var iables 257

conc urrent

clockser ver 219

direc tor y traversal247

echo ser ver 222

non-blo cking cache 272

we b craw ler 239

conﬁnement,ser ial 262

conﬁnement,var iable 261

consistenc y,sequential 268, 269

const de clarat ion14, 75

cons tant

false boole an 63

generator, iota xiii,77

time.Minute 76

time.Second 164

true boole an 63

types, untyped78

cons tants, pre cisionof78

cons traints, bui ld 296

cont ent ion,lock267, 272

cont ext swit c h 280

continue st atement 24, 46

continue st atement,lab ele d 249

cont rac ts, interfaces as 171

cont rol ﬂow 46

conv ersion

bidirec tion altounidirec tion al

ch annel 231

byte slice tostr ing 73

ch arac ter 71

implicit79

narrow ing 40, 55

numeric79

op erat ion40, 55, 64, 71, 78, 79,

173, 187, 194, 208, 231, 353, 358

rune slice tostr ing 71

rune tostr ing 71

st ring71

st ringtobyteslice 40, 73

st ringtoruneslice 71, 88

unsafe.Pointer 356

copy built-in function89

countdown example244, 245, 246

counting semaphore 241

coverage, statement 318, 320

coverage, test 318

coverage_test example319

CPU proﬁle 324

crawl example240, 242, 243

craw ler,con cur rentweb 239

craw ler,web 119

cr iticalsec tion 263, 270, 275

cross-comp ilation 295

cr yptography 55, 83, 121, 325

crypto/sha256 package83

customSort example190

cyclic dat a st ruc ture337

cyclic testdep endency 314

data

race 259, 267, 275

st ruc ture, cyc lic 337

st ruc ture, rec ursive101, 102, 107

type,bit-s et 77

database driver, MySQL284

database/sql package211, 288

daysAgo func tion114

deadbeef 55, 80

de adlo ck233, 240, 265

de clarat ion

const 14, 75

func 3, 29, 119

import 3, 28, 42, 284, 285, 362

method 40, 155

package 2, 28, 41, 285

package-le vel 28

scop e 45, 137

sh adowing 46, 49, 206, 212

short var iable 5, 7, 30, 31

st atement,short var iable 7

struct 99

type 39

var 5, 30

de clarat ions,order of48

de code example, S-expression347

de coder API, token-b ased 213, 215,

347

de coding, S-expression344

de coding, XML 213

de crementstatement -- 5, 37

dedup example97

de ep equivalence 87, 317, 358

default case in select 246

default case in switch 23

default case intyp e switch 212

defer anonymou s func tion146

defer example150, 151

defer st atement 144, 150, 264

defer red functioncal l 144

delete built-in function94

depth-ﬁrs t search algor it hm136

dereference,imp licit159

di agram

helloworld su bst ring69

pip eline228

slice cap acity growt h 90

slice ofmonths84

st ringsharing 65

st ruc t hole 355

thumbnail sequence 238

dig italartifac t example178

Dijkstra, Edsger318

Di lbert 100

direc ted acyclic graph136, 284

direc tor y traversal, con cur rent247

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

370 INDEX

discriminated union211, 213, 214

Display func tion333

display example333

display func tion334

displaying met hodsofatyp e 351

Distance func tion156

do c comment42, 296

doc.go do c comment ﬁle 42, 296

do cumentation,package296

domain name, imp ort pat h 284

dot . in templ ate 113

down loadingpackages 292

Dr.Strange lov e 336

drainingachannel 229, 252

du example247, 249, 250

dup example9,11, 12

dup lic atesup pression276

dy namic disp atch 183

dy namic typ e,int erface 181

echo example5, 7, 34, 309

echo test 309

echo ser ver,con cur rent222

echo_test.go 310

ef fec tive tests, writing 316, 317

emai l client312

embarrassinglyparal lel235

embedde d st ruc t ﬁeld161

embedding, int erface 174

embedding, str uct 104, 161

Employee st ruc t 100

empt y

interface typ e 176

select st atement 245

st ring5, 7, 30

st ruc t 102

encapsulat ion168, 284

encoding API213, 340

enco ding, S-expression338

encoding/json package107

encoding/xml package107, 213

endof ﬁle (EOF) 131

enum 77

enviro nment var iable

GOARCH 292, 295

GOMAXPROCS 281, 321

GOOS 292, 295

GOPATH xv i,291, 295

GOROOT 292

equal func tion87, 96

equality,point er32

equivalence,deep87, 317, 358

error built-in interface 196

error built-in typ e 11, 128, 149, 196

er ror API 127, 152

error.Error method 196

errorf func tion143

er ror-handlingstrateg ies 128, 152,

310, 316

errors package196

errors.New func tion196

es cap e

hexade cimal 66

HTML 116

octal66

sequence 10

sequences, table of66

Unico de 68, 107

URL 111

es capingvar iables 36

eval example198

eventmultiplexing244

events 227, 244

Example func tion302, 326

example

autoescape 117

basename 72

boiling 29

ByteCounter 173

bzipper 365

cf 43

charcount 98

chat 254

clock 220, 222

ColoredPoint 161

comma 73

countdown 244, 245, 246

coverage_test 319

crawl 240, 242, 243

customSort 190

dedup 97

defer 150, 151

dig italartifac t 178

display 333

du 247, 249, 250

dup 9, 11, 12

echo 5, 7, 34, 309

eval 198

fetch 16, 148

fetchall 18

findlinks 122, 125, 139

ftoc 29

github 110, 111

graph 99

helloworld 1, 2

http 192, 194, 195

intset 166

issues 112

issueshtml 115

issuesreport 114

jpeg 287

lissajous 14, 22, 35

mandelbrot 62

memo 275, 276, 277, 278, 279

methods 351

movie 108, 110

netcat 221, 223, 227

netflag 78

nonempty 92

outline 123, 133

package, bank 258, 261, 263

package, bzip 363

package, format 332

package, geometry 156

package, http 192

package, links 138

package, memo 273

package, params 348

package, storage 312, 313

package, tempconv 42

package, thumbnail 235

palindrom e 303, 305, 308

params 348

Parse 152

pipeline 228, 230, 231

pl aylist187

rev 86

reverb 223, 224

server 19, 21

sexpr 340

S-expressiondecode347

sha256 83

sleep 179

spinner 218

squares 135

sum 142

surface 59, 203

tempconv 39, 180, 289

temp eratureconversion29

tempflag 181

test of word 303

thumbnail 236, 237, 238

title 153

topoSort 136

trace 146

treesort 102

urlvalues 160

wait 130

word 303, 305, 308

xmlselect 215

appendInt 88

exception 128, 149

excessive con cur rency 241, 242

exclusion, mut ual 262, 267

exclusivelock263, 266, 270

exclusiveORoperator ^ 53

exp onent ial back-off 130

exp ort ofstr uct ﬁeld101, 106, 109,

110, 168

export_test.go ﬁle 315

Expr.Check method 202

expression

addressable159, 341

evaluator197

method 164

re ceive 225

Expr.Eval method 199

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

INDEX 371

extendingaslice 86

Extensible Marku p Language (XML)

107

exter nal testpackage285, 314

Fahrenheit type 39

fai luremessage , test 306

fallthrough st atement 23, 212

false boole an cons tant63

fetch example16, 148

fetchall example18

fib func tion37, 218

Fibonacci algor it hm37, 218

ﬁeld

anonymou s st ruc t 104, 105, 106,

162

embedde d st ruc t 161

exp ort ofstr uct 101, 106, 109,

110, 168

order,str uct 101, 355

sele ctor156

st ruc t 15, 99

tag, omitempty 109

tag, str uct 109, 348

ﬁgure

Liss ajous 13

Mandelbro t 63

3-D sur face 58, 203

Fi le Transfer Pro tocol (FTP) 222

ﬁle

block46

export_test.go 315

name,Micros oft Windows 72

name,POSIX 72

_test.go 285, 302, 303

findlinks example122, 125, 139

ﬁxe d-size stack 124

flag package33, 179

ﬂag

go tool -bench 321

go tool -benchmem 322

go tool -covermode 319

go tool -coverprofile 319

go tool -cpuprofile 324

go tool -nodecount 325

go tool -text 325

go tool -web 326

godoc -analysis 176

go list -f 315

go -race 271

go test -race 274

go test -run 305

go test -v 304

flag.Args func tion34

flag.Bool func tion34

flag.Duration func tion179

flag.Parse func tion34

flag.String func tion34

flag.Value interface 179, 180

ﬂoating-p oint

numb er56

precision56, 57, 63, 78

tr unc ation 40, 55

fmt package2

fmt.Errorf func tion129, 196

fmt.Fprintf func tion172

fmt.Printf func tion10

fmt.Println func tion2

fmt.Scanf func tion75

fmt.Sscanf func tion180

fmt.Stringer interface 180, 210

for scop e 47

for st atement 6

forEachNode func tion133

foreig n-functionint erface (FFI) 361

format, code 3, 6, 9, 48

format examplepackage332

formatAtom func tion332

framewor k,web 193

ftoc example29

func de clarat ion3,29, 119

func tion

anonymou s 22, 135, 236

append built-in 88, 90, 91

argument119

assertion316

Benchmark 302, 321

body, missing121

breadthFirst 139

btoi 64

bufio.NewReader 98

bufio.NewScanner 9

bufio.ScanWords 99

bytes.Equal 86

call,defer red 144

call, ok value fro m 128

cap built-in 84, 232

CelsiusFlag 181

CheckQuota 312, 313

close built-in 226, 228, 251

cmplx.Sqrt 61

comp arison133

complex built-in 61

copy built-in 89

daysAgo 114

delete built-in 94

Display 333

display 334

Distance 156

equal 87, 96

errorf 143

errors.New 196

Example 302, 326

fib 37, 218

flag.Args 34

flag.Bool 34

flag.Duration 179

flag.Parse 34

flag.String 34

fmt.Errorf 129, 196

fmt.Fprintf 172

fmt.Printf 10

fmt.Println 2

fmt.Scanf 75

fmt.Sscanf 180

forEachNode 133

formatAtom 332

gcd 37

hand ler 19, 21, 152, 191, 194, 195,

348

html.Parse 121, 125

http.DefaultServeMux 195

http.Error 193

http.Get 16, 18

http.Handle 195

http.HandleFunc 19, 22, 195

http.ListenAndServe 19, 191

http.NewRequest 253

http.ServeMux 193

hypot 120

imag built-in 61

image.Decode 288

image.RegisterFormat 288

incr 33

init 44, 49

intsToString 74

io.Copy 17, 18

ioutil.ReadAll 16, 272

ioutil.ReadDir 247

ioutil.ReadFile 12, 145

io.WriteString 209

itob 64

json.Marshal 108

json.MarshalIndent 108

json.NewDecoder 111

json.NewEncoder 111

json.Unmarshal 110, 114

len built-in 4, 54, 64, 65, 81, 84,

233

links.Extract 138

literal 22, 135, 227

log.Fatalf 49, 130

main 2, 310

make built-in 9, 18, 88, 94, 225

math.Hypot 156

math.Inf 57

math.IsInf 57

math.IsNaN 57

math.NaN 57

mu lti-value d 11, 30, 37, 96, 125,

126

mustCopy 221

net.Dial 220

net.Listen 220

new built-in 34

ni l 132

os.Close 11

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

372 INDEX

os.Exit 16, 34, 48

os.Getwd 48

os.IsExist 207

os.IsNotExist 207

os.IsPermission 207

os.Open 11

os.Stat 247

panic built-in 148, 149

parameter 119

params.Unpack 349

png.Encode 62

PopCount 45

real built-in 61

recover built-in 152

re cursive anony mou s 137

reflect.TypeOf 330

reflect.ValueOf 331, 337

reflect.Zero 345

regexp.Compile 149

regexp.MustCompile 149

resu ltlist119

runtime.Stack 151

SearchIssues 111

sexpr.Marshal 340

sexpr.readList 347

sexpr.Unmarshal 347

sig nature120

sort.Float64s 191

sort.Ints 191

sort.IntsAreSorted 191

sort.Reverse 189

sort.Strings 95, 137, 191

Sprint 330

sqlQuote 211, 212

strconv.Atoi 22, 75

strconv.FormatInt 75

strconv.Itoa 75

strconv.ParseInt 75

strconv.ParseUint 75

strings.Contains 69

strings.HasPrefix 69

strings.HasSuffix 69

strings.Index 289

strings.Join 7, 12

strings.Map 133

strings.NewReader 289

strings.NewReplacer 289

strings.Split 12

strings.ToLower 72

strings.ToUpper 72

template.Must 114

template.New 114

Test 302

time.After 245

time.AfterFunc 164

time.Now 220

time.Parse 220

time.Since 114

time.Tick 244, 246

title 144, 145

type 119, 120

unicode.IsDigit 71

unicode.IsLetter 71

unicode.IsLower 71

unicode.IsSpace 93

unicode.IsUpper 71

unsafe.AlignOf 355

unsafe.Offsetof 355

unsafe.Sizeof 354

url.QueryEscape 111

utf8.DecodeRuneInString 69

utf8.RuneCountInString 69

value 132

var iadic 142, 172

visit 122

WaitForServer 130

walkDir 247

zerovalue 132

garb agecol lec tion xi, xiii, 7, 35, 230,

353, 357

garb agecol lec tor,mov ing 357

GCD algor it hm37

gcd func tion37

geometry examplepackage156

geometry.Point.Distance method

156

gett ermet hod 169

GIF animat ion13

GitHub issuetracker110

github example110, 111

Pl ayg round xvi,326

Blog xvi, 326

issue110, 112, 358

go to ol2,42, 44, 290

go tool -bench ﬂag 321

go tool -benchmem ﬂag 322

go tool -covermode ﬂag 319

go tool -coverprofile ﬂag 319

go tool -cpuprofile ﬂag 324

go tool -nodecount ﬂag 325

go tool pprof 325

go tool -text ﬂag 325

go tool -web ﬂag 326

go tool cover 318, 319

go doc to ol25

go st atement 18, 218

GOARCH enviro nment var iable 292,

295

go build 2, 286, 293, 294

go doc 296

godoc -analysis ﬂag 176

godoc to olxvi,25, 297, 326

go env 292

gofmt to ol3, 4, 44, 286

go get xv i, 2, 292, 293

go help 290

goimports to ol3,44, 286

go install 295

golang.org/x/net/html package

122

golint to ol292

go list 298, 315

go list -f ﬂag 315

GOMAXPROCS enviro nment var iable

281, 321

GOOS enviro nment var iable 292, 295

GOPATH enviro nment var iable xvi,

291, 295

gopl.io repository xvi

go -race ﬂag 271

GOROOT enviro nment var iable 292

goro utine 18, 217, 233, 235

clos er 238, 250

identity 282

le ak 233, 236, 246

monitor261, 277

mu ltiplexing281

vs. OS thread280

go run 2, 294

go test 301, 302, 304

go test -race ﬂag 274

go test -run ﬂag 305

go test -v ﬂag 304

goto st atement 24

graph example99

GraphViz 326

Gr ies emer,Rob ert xi

growth,stack 124, 280, 358

gu ardingmut ex263

half-openint erval 4

hand ler function19, 21, 152, 191,

194, 195, 348

‘‘happ ens before’’ re lat ion226, 257,

261, 277

‘‘hasa’’ re lat ions hip 162

hash table 9, 93

Haskel l prog ramminglangu agexiv

he ap

al location 36

proﬁle 324

var iable 36

helloworld example1,2

helloworld su bst ringdiagram 69

hexade cimal escape 66

hexade cimal literal 55

hidden point er357

Ho are , To nyxiii

hole,str uct 354

HTML

anch orelement122

es cap e 116

injec tion att ack 115

metach arac ter 116

pars er121

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

INDEX 373

html.Parse func tion121, 125

html/template package113, 115

HT TP

GETrequest21, 127, 272, 348

POSTrequest348

re quest, cancellat ionof253

re questmultiplexer 193

http example192, 194, 195

http examplepackage192

(*http.Client).Do method 253

http.DefaultClient var iable 253

http.DefaultServeMux func tion

195

http.Error func tion193

http.Get func tion16, 18

http.Handle func tion195

http.HandleFunc func tion19, 22,

195

http.Handler interface 191, 193

http.HandlerFunc type 194, 203

http.ListenAndServe func tion19,

191

http.NewRequest func tion253

http.Request type 21, 253

(*http.Request).ParseForm

method 22, 348

http.ResponseWriter type 19, 22,

191, 193

http.ServeMux func tion193

hypot func tion120

identiﬁer _,blank 7,38, 95, 120, 126,

287

identiﬁer,qualiﬁed 41, 43

identity,goroutine 282

IEEE 754 stand ard 56, 57

if,initializat ionstatement in22,

206

if-else scop e 47

if-else st atement 9,22, 47

imag built-in function61

imagemanipu lat ion121

image package62, 287

image/color package14

image.Decode func tion288

image/png package288

image.RegisterFormat func tion

288

imag inar y literal 61

immut abi lit y 261

immut abi lit y,str ing 65, 73

implementation wit h slice,stack 92,

215

implicit

& 158, 167

assig nment38

conv ersion79

dereference 159

import de clarat ion3,28, 42, 284,

285, 362

import

bl ank 287

path 284

path dom ain name284

renaming286

incr func tion33

incrementstatement ++ 5, 37, 94

index operat ion, str ing 64

indirec tion operator * 24, 32

inﬁnite loop6,120, 228

infor mat ionhiding168, 284

init func tion44, 49

initializat ion

lazy 268

package44

st atement in if 22, 206

st atement in switch 24

initializer list 30

injec tion att ack,HTML115

injec tion att ack,SQL 211

in-place slice techniques 91

insertionsor t algor it hm101

int type 52

integer

literal 55

ov erﬂow 53, 113

sig ned 52, 54

unsig ned 52, 54

integrat iontest314

interface

assig nabilit y 175

comp arison184

dy namic typ e 181

embedding174

error built-in 196

flag.Value 179, 180

fmt.Stringer 180, 210

http.Handler 191, 193

io.Closer 174

io.Reader 174

io.Writer 15, 22, 172, 174, 186,

208, 209, 309

JSON110

method cal l 182

ni l 182

pit fal l 184

ReadWriteCloser 174

ReadWriter 174

satisfac tion171, 175

sort.Interface 186

type 171, 174

interface{} type 143, 176, 331

interface

type ass ertion208, 210

type,emp ty176

value 181

with nil point er184

zerovalue 182

interfaces as cont rac ts 171

internal package298

intset example166

intsToString func tion74

invar iants 159, 169, 170, 265, 284,

311, 352

io package174

io.Closer interface 174

io.Copy func tion17, 18

io.Discard st ream22

io.Discard var iable 18

io.EOF var iable 132

io/ioutil package16, 145

io.Reader interface 174

iota cons tantgenerator xiii,77

ioutil.ReadAll func tion16, 272

ioutil.ReadDir func tion247

ioutil.ReadFile func tion12, 145

io.Writer interface 15, 22, 172,

174, 186, 208, 209, 309

io.WriteString func tion209

‘‘is a’’ re lat ions hip 162, 175

issue, Go110, 112, 358

issuetracker, GitHu b 110

issues example112

issueshtml example115

issuesreport example114

it erat ionorder,map 95

it erat ionvar iable,capturing 140

itob func tion64

Java programminglangu agexv

JavaS crip t Objec t No tat ion(JSON)

107, 338

JavaS crip t prog ramminglangu age

xv,107

jpeg example287

JSON

interface 110

interface,OpenMov ieDat abas e

113

interface,xkcd113

mars haling108

unmarsh aling110

json.Decoder type 111

json.Encoder type 111

json.Marshal func tion108

json.MarshalIndent func tion108

json.NewDecoder func tion111

json.NewEncoder func tion111

json.Unmarshal func tion110, 114

ke yword , type 212

ke yword s,table of27

Knut h,Don ald 323

labelscope 46

label, statement 46

labele d

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

374 INDEX

break st atement 249

continue st atement 249

st atement 46

layout,memor y 354, 355

lazy initializat ion268

le ak, goro utine 233, 236, 246

lef t shif t op erator << 54

len built-in function4,54, 64, 65,

81, 84, 233

lexic al block46, 120, 135, 141, 212

lexic al closure136

lifet ime, var iable 35, 46, 135

links examplepackage138

links.Extract func tion138

Lisp programminglangu age338

Liss ajous algor it hm15

Liss ajous ﬁgure13

lissajous example14, 22, 35

list,initializer 30

literal

ar ray 82, 84

comp osite 14

func tion22, 135, 227

hexade cimal 55

imag inar y 61

integer55

map94

octal55

raw str ing 66

rune 56

slice 38, 86

st ring65

st ruc t 15, 102, 106

lo cal

block46

var iable 29, 141

var iable,addressof32, 36

var iable scope 135

lo cat ingpackages 291

lo ck

cont ent ion 267, 272

exclusive263, 266, 270

mu tex 102, 263, 264, 324

non-reent rant265

re aders266

sh are d 266

wr iter266

log package49, 130, 170

log.Fatalf func tion49, 130

lo oku p m[key],map 94

lo oku p, ok value fro m map96

lo op

inﬁnite 6,120, 228

range 6, 9

var iable,capturing 141, 236, 240

var iable scope 141, 236

while6

main func tion2,310

main,package2,285, 310

make built-in function9,18, 88, 94,

225

make ch annel 18, 225

make map9,18, 94

make slice 88, 322

Mandelbro t ﬁgure63

Mandelbro t set61

mandelbrot example62

map

as set 96, 202

comp arison96

element, non existent94, 95

it erat ionorder 95

literal 94

lo oku p m[key] 94

lo oku p, ok value fro m 96

make 9, 18, 94

ni l 95

range ov er94

type 9,93

with slice key 97

zerovalue 95

mars halingJSON108

math package14, 56

math/big package63

math/cmplx package61

math.Hypot func tion156

math.Inf func tion57

math.IsInf func tion57

math.IsNaN func tion57

math.NaN func tion57

math/rand package285, 308

memo example275, 276, 277, 278,

279

memo examplepackage273

memoizat ion272

memory allocat ion36, 71, 89, 169,

209, 322

memory layout354, 355

metach arac ter,HTML116

method

(*bufio.Reader).ReadRune 98

(*bufio.Scanner).Err 97

(*bufio.Scanner).Scan 9

(*bufio.Scanner).Split 99

(*bytes.Buffer).Grow 169

(*bytes.Buffer).WriteByte 74

(*bytes.Buffer).WriteRune 74

(*bytes.Buffer).WriteString

call,int erface 182

ch aining114

de clarat ion40, 155

error.Error 196

Expr.Check 202

expression164

Expr.Eval 199

geometry.Point.Distance 156

gett er169

(*http.Client).Do 253

(*http.Request).ParseForm 22,

348

name 156

net.Conn.Close 220

net.Listener.Accept 220

(*os.File).Write 183

path.Distance 157

prom otion 161

re ceivername157

re ceiverparameter 156

re ceivertyp e 157

reflect.Type.Field 348

reflect.Value.Addr 342

reflect.Value.CanAddr 342

reflect.Value.Interface 331,

342

reflect.Value.Kind 332

sele ctor156

sett er169

String 40, 166, 329

(*sync.Mutex).Lock 21, 146, 263

(*sync.Mutex).Unlock 21, 146,

263

(*sync.Once).Do 270

(*sync.RWMutex).RLock 266

(*sync.RWMutex).RUnlock 266

(*sync.WaitGroup).Add 238

(*sync.WaitGroup).Done 238

template.Funcs 114

template.Parse 114

(*testing.T).Errorf 200, 304,

306

(*testing.T).Fatal 306

time.Time.Format 220

value 164

(*xml.Decoder).Token 213

methods example351

methodsofatyp e,displ aying 351

Micros oft Windows ﬁle name72

missingfunctionbody121

m[key],map lookup 94

mobi leplatfor ms121

Mo dula-2 programminglangu age

xiii

mo dular ity 283

monitor264, 275

monitorgoroutine 261, 277

movie example108, 110

moving garb agecol lec tor 357

mu ltimap160, 193

mu ltiple-value assig nment37

mu ltiplexer,HTTPrequest193

mu ltiplexing, event 244

mu ltiplexing, goroutine 281

mu ltithre ading, share d-memor y

217, 257

mu lti-value d func tion11, 30, 37, 96,

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

INDEX 375

125, 126

mustCopy func tion221

mu tex 145, 163, 256, 269

gu arding263

lo ck102, 263, 264, 324

re ad/w rit e 266, 267

mu tualexc lusion262, 267

MySQLdat abas e dr iver284

name

method 156

method receiver157

package28, 43

parameter 120

sp ace 41, 156, 283

name d

resu lt120, 126

resu ltzerovalue 120, 127

type 24, 39, 40, 105, 157

namingconvent ion 28, 169, 174,

289

naming, package289

NaN(notanumber) 57, 93

narrow ing conversion40, 55

negat ionoperator ! 63

net package219

netcat example221, 223, 227

net.Conn type 220

net.Conn.Close method 220

net.Dial func tion220

netflag example78

net/http package16, 191

net.Listen func tion220

net.Listener type 220

net.Listener.Accept method 220

net/smtp package312

net/url package160

networ king121, 219

new built-in function34

new,redeﬁning35

ni l

ch annel 246, 249

func tion132

interface 182

map95

pointer32

pointer, int erface wit h 184

re ceiver159, 185

slice 87

non-blo cking

cach e 275

cach e, conc urrent272

ch annel receive 246

select 246

nonempty example92

nonexistentmap element94, 95

non-reent rantlock265

non-stand ard package121

numb er, ﬂoating-p oint56

numb erzerovalue 5, 30

numeric

conv ersion79

precision55, 78

type 51

Ob ero n prog ramminglangu agexiii

objec t 156

objec t-oriente d prog ramming

(O OP) 155, 168

octalescap e 66

octallit eral 55

ok value 37

ok value fro m ch annel receive 229

ok value fro m func tioncal l 128

ok value fro m maplooku p 96

ok value fro m type ass ertion206

omitempty ﬁeldtag 109

Op enMov ieDat abas e JSON

interface 113

op erat ion, atomic 264

op erat ion, conv ersion40, 55, 64, 71,

78, 79, 173, 187, 194, 208, 231,

353, 358

op erator

+=, -=,etc., assig nment5

&,address-of24, 32, 94, 158, 167

&^,AND-NOT 53

&^,bit-cle ar 53

^,bit w ise complement53

|,bit w ise OR166, 167

==,comparison40, 63

^,exc lusiveOR53

*,indirec tion 24, 32

<<,lef t shif t 54

!,negat ion63

%,remainder 52, 166

>>,rig htshif t 54

&&,short-circuit AND 63

||,short-circuit OR63

+,str ing con catenat ion5,65

-,unary 53

+,unary 53

asso ciativity 52

precedence 52, 63

s[i:j],slice 84, 86

s[i:j],subst ring65, 86

op erator s

assig nment36, 52

comp arison40, 93

tableofbinar y 52

tableofbit w ise 53

tableofcomparison53

opt imizat ion264, 321, 323

opt imizat ion, premature 324

OR operator ||,short-circuit 63

order ofdeclarat ions 48

order,str uct ﬁeld101, 355

or ganizat ion, wor ksp ace 291

OS threadvs. goroutine 280

os package4,206

os.Args var iable 4

os.Close func tion11

os.Exit func tion16, 34, 48

*os.File type 11, 13, 172, 175, 185,

336

os.FileInfo type 247

(*os.File).Write method 183

os.Getwd func tion48

os.IsExist func tion207

os.IsNotExist func tion207

os.IsPermission func tion207

os.LinkError type 207

os.Open func tion11

os.PathError type 207

os.Stat func tion247

outline example123, 133

// Output comment326

ov erﬂow,int eger53, 113

ov erﬂow,stack 124

package de clarat ion2,28, 41, 285

package

API284, 296, 311, 333, 352

back-do or315

bank example258, 261, 263

block46

bufio 9

bytes 71, 73

bzip example363

compress/bzip2 361

crypto/sha256 83

database/sql 211, 288

do cumentation 296

encoding/json 107

encoding/xml 107, 213

errors 196

exter nal test285, 314

flag 33, 179

fmt 2

format example332

geometry example156

golang.org/x/net/html 122

html/template 113, 115

http example192

image 62, 287

image/color 14

image/png 288

initializat ion44

internal 298

io 174

io/ioutil 16, 145

links example138

log 49, 130, 170

main 2, 285, 310

math 14, 56

math/big 63

math/cmplx 61

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

376 INDEX

math/rand 285, 308

memo example273

name 28, 43

naming289

net 219

net/http 16, 191

net/smtp 312

net/url 160

non-stand ard 121

os 4, 206

params example348

path 72

path/filepath 72

reflect 330

regexp 149

runtime 151

sort 95, 186, 189

storage example312, 313

strconv 22, 71, 75

strings 7, 71, 72, 289

sync 237, 263

syscall 196, 208

tempconv example42

testing 285, 302

text/scanner 344

text/tabwriter 188

text/template 113, 300

thumbnail example235

time 18, 77, 183

unicode 71

unicode/utf8 69

unsafe 354

package-le vel declarat ion28

packages

building293

down loading292

lo cat ing291

quer ying298

palindrom e 191

palindrom e example303, 305, 308

panic 64, 152, 253

panic built-in function148, 149

paradoxic al race 267

paral lelcomposition 224

paral lel, emb arrassingly235

paral lelism 217

parameter

... 91, 142, 143, 172

func tion119

method receiver156

name 120

passing120

unus ed120

params example348

params examplepackage348

params.Unpack func tion349

parent hes es 4, 6, 9, 52, 63, 119, 146,

158, 285, 335, 345

Parse example152

pars er, HTML121

Pascal programminglangu agexiii

path, ... 292, 299

path package72

path.Distance method 157

path/filepath package72

Pi ke,Rob xi, xiii,67, 107

pipeline example228, 230, 231

pip eline227

pip elinediagram 228

pit fal l,int erface 184

pit fal l,scope 140

pl atfor ms, mobile121

Pl ayg round,Goxvi,326

pl aylistexample187

png.Encode func tion62

pointer24, 32, 34

ali asing33

argument33, 83

equality 32

hidden 357

ni l 32

re ceiver158, 167

to str uct 100, 103

zerovalue 32

pollingchannel 246

poly mor phism, ad hoc 211

poly mor phism, subtype 211

PopCount func tion45

Po rtableNet wor k Graphics (PNG)

POSIX ﬁle name72

POSIX stand ard xi, 55, 72, 197

precedence,operator 52, 63

precision

ﬂoating-p oint56, 57, 63, 78

numeric55, 78

of con stants78

predeclare d names, table of28

prematureabstrac tion216, 316, 317

prematureopt imizat ion324

Printf %% 10

Printf verbs, table of10

Printf %b 10, 54, 75

Printf %c 10, 56

Printf %d 10, 55

Printf %e 10, 57

Printf %f 10, 57

Printf %g 10, 57

Printf %[n] 56

Printf %o 10, 55

Printf %q 10, 56, 97

Printf %s 10

Printf %*s 134

Printf %T 10, 80, 83, 184, 331

Printf %t 10, 83

Printf %#v 106, 207

Printf %v 10, 11

Printf % x 71

Printf %#x 56

Printf %x 10, 55, 83

produc tioncode301

proﬁle

blocking 324

CPU 324

he ap324

proﬁling324

prog ramminglangu age

Alef xiii

APLxiii

C++ xiv, xv, 361

Cxii,xv, 1, 6, 52, 260, 361

Haskel l xiv

Java xv

JavaS crip t xv,107

Lisp 338

Mo dula-2 xiii

Ob ero n xiii

Pascal xiii

Python xv, 193

Ru byxv, 193

Scheme xiii

Sque ak, Ne wsque ak xiii

prom otion,met hod 161

prot ocolbuf fers107

Python prog ramminglangu agexv,

193

qu aliﬁed identiﬁer 41, 43

quer yingpackages 298

quot e ch arac ter, ' 56

race

condit ion 21, 257, 258, 259

detec tor 271, 274

paradoxic al 267

randomize d test ing 307

range lo op6,9

range ov erchannel 229

range ov ermap 94

range ov erstr ing 69, 88

{{range}} temp lateaction113

raw str ing lit eral 66

re ach abi lit y 36

re ad, stale 268

re aderslock266

re ad/w rit e mu tex 266, 267

ReadWriteCloser interface 174

ReadWriter interface 174

real built-in function61

re ceive

<-ch,channel 18, 225, 232

expression225

non-blo cking channel 246

ok value fro m ch annel 229

re ceive-onlychannel typ e <-chan T

230

re ceiver

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

INDEX 377

name,met hod 157

ni l 159, 185

parameter,met hod 156

pointer158, 167

type,met hod 157

recover built-in function152

re cursion121, 124, 247, 333, 339,

345, 359

re cursive

anonymou s func tion137

data str ucture101, 102, 107

type 48

re deﬁning new 35

reference

call by83

identity 87

type 9,12, 93, 120

reflect package330

reﬂec tion 329, 352, 359

reflect.StructTag type 348

reflect.Type type 330

reflect.Type.Field method 348

reflect.TypeOf func tion330

reflect.Value type 331, 342

reflect.Value zerovalue 332

reflect.Value.Addr method 342

reflect.Value.CanAddr method

342

reflect.Value.Interface method

331, 342

reflect.Value.Kind method 332

reflect.ValueOf func tion331, 337

reflect.Zero func tion345

regexp package149

regexp.Compile func tion149

regexp.MustCompile func tion149

regu lar expression66, 149, 305, 321

re lat ion, ‘‘happ ens before’’ 226, 257,

261, 277

re lat ions hip, ‘‘hasa’’ 162

re lat ions hip, ‘‘is a’’ 162, 175

remainder operator % 52, 166

renamingimp ort 286

rendezvous 234

repl acementcharac ter (,Unico de

70, 98

repository, gopl.io xv i

re quest

HT TP GET21, 127, 272, 348

HT TP POST348

mu ltiplexer,HTTP193

resu ltlist, function119

resu lt, named 120, 126

return,bare126

return st atement 29, 120, 125

rev example86

reverb example223, 224

rightshif t op erator >> 54

Ru byprogramminglangu agexv,

193

rune literal 56

rune type 52, 67

rune slice tostr ing conversion71

rune tostr ing conversion71

runtime package151

runt ime API 324

runt ime sch edu ler 281

runtime.Stack func tion151

satisfac tion, interface 171, 175

Scal able Vec tor Graphics (SVG) 58

sche duler,runtime 281

Scheme programminglangu agexiii

scop e

de clarat ion45, 137

for 47

if-else 47

label46

lo cal variable135

lo opvar iable 141, 236

pit fal l 140

short var iable decl arat ion22, 48

switch 47

search algor it hm, bre adt h-ﬁrs t 139,

239

search algor it hm, depth-ﬁrs t 136

SearchIssues func tion111

select case 245

select, default case in246

select,non-blo cking 246

select st atement 244, 245

select{} st atement 245

sele ctive recov ery 152

sele ctor, ﬁeld156

sele ctor, met hod 156

semaph ore , binary 262

semaph ore , counting 241

semicolon3,6

send ch<-,channel 18, 225, 232

send statement 225

send-onlychannel typ e chan<- T

230

separatecompi lat ion284

sequence diag ram, thumbnail 238

sequential consistenc y 268, 269

serial conﬁnement 262

server example19, 21

server

ch at253

conc urrentclo ck219

conc urrentech o 222

set, map as 96, 202

sett ermet hod 169

sexpr example340

S-expression

de code example347

de coding344

enco ding338

sexpr.Marshal func tion340

sexpr.readList func tion347

sexpr.Unmarshal func tion347

SHA256 message digest83

sha256 example83

sh adowing declarat ion46, 49, 206,

212

sh are d

lo ck266

var iables 257

var iables, conc urrency wit h 257

sh are d-memor y mu ltithre ading217,

257

shif t op erator <<,lef t 54

shif t op erator >>,rig ht54

short

var iable decl arat ion5, 7, 30, 31

var iable decl arat ionscope 22, 48

var iable decl arat ionstatement 7

short-circuit

AND operator && 63

evaluation 63

OR operator || 63

sig nature, function120

sig ned int eger52, 54

s[i:j],slice operator 84, 86

s[i:j],subst ringoperator 65, 86

simplestatement 6,22

Sizeof table354

sleep example179

slice 4

argument86

capacity 88, 89

capacity growt h di agram 90

comp arison87

extending a 86

ke y,map wit h 97

literal 38, 86

make 88, 322

ni l 87

of monthsdiagram 84

op erator s[i:j] 84, 86

ro tat ionalgor it hm86

te chniques, in-place 91

type 84

us ed as stack 123

zerolengt h 87

zerovalue 74, 87

SMTP client312

so cket

TCP219

UDP219

Unix domain 219

sort algor it hm, top olog ical 136

sort package95, 186, 189

sort.Float64s func tion191

sort.Interface interface 186

sort.Ints func tion191

sort.IntsAreSorted func tion191

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

378 INDEX

sort.IntSlice type 191

sort.Reverse func tion189

sort.Strings func tion95, 137, 191

spinner example218

Sprint func tion330

SQLAPI 211

SQLinj e ction att ack 211

sqlQuote func tion211, 212

squares example135

Sque ak, Ne wsque ak prog ramming

language xiii

st ack

al location 36

ﬁxe d-size 124

growth 124, 280, 358

implementation wit h slice 92, 215

ov erﬂow 124

slice usedas123

trace 149, 253

var iable 36

var iable-size 124

st ale read268

st and ard

IEEE 754 56, 57

POSIX xi,55, 72, 197

Unico de 2, 27, 52, 66, 67, 69, 97

st atement

--,decrement 5,37

++,increment 5,37, 94

assig nment5, 7, 36, 52, 94, 173

break 24, 46

continue 24, 46

coverage318, 320

defer 144, 150, 264

fallthrough 23, 212

for 6

go 18, 218

goto 24

if-else 9, 22, 47

label46

labele d 46

return 29, 120, 125

select{} 245

select 244, 245

send 225

short var iable decl arat ion7

simple6,22

switch 23, 47

tagless switch 24

type switch 210, 212, 214, 329

unreach able120

storage examplepackage312, 313

St range lov e,Dr. 336

st rateg ies, er ror-handling128, 152,

310, 316

strconv package22, 71, 75

strconv.Atoi func tion22, 75

strconv.FormatInt func tion75

strconv.Itoa func tion75

strconv.ParseInt func tion75

strconv.ParseUint func tion75

st ream, io.Discard 22

String method 40, 166, 329

st ring

conc atenation operator + 5, 65

conv ersion71

immut abi lit y 65, 73

index operat ion64

literal 65

literal,raw 66

range ov er69, 88

sh aring diagram 65

test 71

to byteslice conv ersion40, 73

to runeslice conv ersion71, 88

zerovalue 5, 7, 30

comp arison65

strings package7,71, 72, 289

strings.Contains func tion69

strings.HasPrefix func tion69

strings.HasSuffix func tion69

strings.Index func tion289

strings.Join func tion7,12

strings.Map func tion133

strings.NewReader func tion289

strings.NewReplacer func tion289

strings.Reader type 289

strings.Replacer type 289

strings.Split func tion12

strings.ToLower func tion72

strings.ToUpper func tion72

struct de clarat ion99

st ruc t

comp arison104

embedding104, 161

Employee 100

empt y 102

ﬁeld15, 99

ﬁeld, anony mou s 104, 105, 106,

162

ﬁeld, emb edde d 161

ﬁeld, exp ort of101, 106, 109, 110,

168

ﬁeldorder 101, 355

ﬁeldtag 109, 348

hole 354

hole diag ram 355

literal 15, 102, 106

literal,addressof103

pointerto100, 103

type 15, 24, 99

struct{} type 227, 241, 250

st ruc t type,unnamed 163

st ruc t zerovalue 102

su bst itutabi lit y 193

su bst ringoperator s[i:j] 65, 86

su btype poly mor phism 211

sum example142

surface example59, 203

surface ﬁgure, 3-D 58, 203

SVG58

SWIG361

Sw iss armyknife290

switch, default case in23

switch,initializat ionstatement in

switch scop e 47

switch st atement 23, 47

switch st atement,tag less24

switch st atement,typ e 210, 212,

214, 329

sw itch,context 280

sync package237, 263

sy nchro nou s ch annel 226

sync.Mutex type 263, 269

(*sync.Mutex).Lock method 21,

146, 263

(*sync.Mutex).Unlock method 21,

146, 263

sync.Once type 270

(*sync.Once).Do method 270

sync.RWMutex type 266, 270

(*sync.RWMutex).RLock method

266

(*sync.RWMutex).RUnlock method

266

sync.WaitGroup type 237, 250, 274

(*sync.WaitGroup).Add method

238

(*sync.WaitGroup).Done method

238

syscall package196, 208

syscall.Errno type 196, 197

systemcal l API196

tableof

binary operator s 52

bit w ise operator s 53

comp arisonoperator s 53

es cap e sequences 66

ke yword s 27

predeclare d names28

Printf verbs 10

UTF-8 enco dings 67

table, Sizeof 354

table-dr iventesting 200, 306, 319

tag, str uct ﬁeld109, 348

tagless switch st atement 24

tags, bui ld 296

TCPsocket219

te chniques, in-place slice 91

tempconv example39, 180, 289

tempconv examplepackage42

temp eratureconversionexample29

tempflag example181

template API115

temp late

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

INDEX 379

| in 113

ac tion, {{range}} 113

dot . in 113

template.Funcs method 114

template.HTML type 116

template.Must func tion114

template.New func tion114

template.Parse method 114

Test func tion302

test

bl ack-b ox310

br ittle317

ch arac ter 71

coverage318

dep endency,cyc lic 314

echo 309

fai luremessage 306

integrat ion314

of word example303

package, exter nal 285, 314

st ring71

whit e-b ox311

assertion306

_test.go ﬁle 285, 302, 303

testing package285, 302

test ing

acommand 308

randomize d 307

table-dr iven200, 306, 319

testing.B type 321

testing.T type 302

(*testing.T).Errorf method 200,

304, 306

(*testing.T).Fatal method 306

tests, writing effec tive 316, 317

text/scanner package344

text/tabwriter package188

text/template package113, 300

Thomps on, Ken xi,67

thre ad218, 280

thre ad-lo cal storage282

3-D sur face ﬁgure58, 203

thumbnail example236, 237, 238

thumbnail examplepackage235

thumbnail sequence diag ram 238

time package18, 77, 183

time.After func tion245

time.AfterFunc func tion164

time.Duration type 76, 179

time.Minute cons tant76

time.Now func tion220

time.Parse func tion220

time.Second cons tant164

time.Since func tion114

time.Tick func tion244, 246

time.Time type 114

time.Time.Format method 220

title example153

title func tion144, 145

token-b ased decoder API213, 215,

347

token-b ased XML deco ding213

to ol

cgo 361, 362

go 2, 42, 44, 290

go doc 25

godoc xv i,25, 297, 326

gofmt 3, 4, 44, 286

goimports 3, 44, 286

golint 292

topolog ical sor t algor it hm136

topoSort example136

trace example146

trace,stack 149, 253

tree,binar y 102

treesort example102

true boole an cons tant63

tr unc ation,ﬂoating-p oint40, 55

tupleassig nment31, 37

type de clarat ion39

type ke yword 212

type

abstrac t 24, 171

ag gregate81, 99

ar ray 81

assertion205, 211

assertion, interface 208, 210

assertion, ok value fro m 206

bool 63

bufio.Scanner 9

byte 52

bytes.Buffer 74, 169, 172, 185

Celsius 39

chan 225

ch annel 18

<-chan T,receive-onlychannel

230

chan<- T,send-onlychannel 230

complex 61

comp osite xv, 14, 81

comp osition xv, 107, 162, 189

concrete 24, 171, 211, 214

displaying met hodsof a 351

empt y interface 176

error built-in 11, 128, 149, 196

Fahrenheit 39

func tion119, 120

http.HandlerFunc 194, 203

http.Request 21, 253

http.ResponseWriter 19, 22,

191, 193

int 52

interface{} 143, 176, 331

interface 171, 174

interface dynamic 181

json.Decoder 111

json.Encoder 111

map9,93

method receiver157

mismatch 55

name d 24, 39, 40, 105, 157

net.Conn 220

net.Listener 220

numeric51

*os.File 11, 13, 172, 175, 185,

336

os.FileInfo 247

os.LinkError 207

os.PathError 207

re cursive 48

reference9,12, 93, 120

reflect.StructTag 348

reflect.Type 330

reflect.Value 331, 342

rune 52, 67

slice 84

sort.IntSlice 191

strings.Reader 289

strings.Replacer 289

struct{} 227, 241, 250

st ruc t 15, 24, 99

switch, case in 212

switch, default case in212

switch st atement 210, 212, 214,

329

sync.Mutex 263, 269

sync.Once 270

sync.RWMutex 266, 270

sync.WaitGroup 237, 250, 274

syscall.Errno 196, 197

template.HTML 116

testing.B 321

testing.T 302

time.Duration 76, 179

time.Time 114

uint 52

uintptr 52, 354, 357

underly ing 39

unidirec tion alchannel 230, 231

unnamed str uct 163

unsafe.Pointer 356

url.URL 193

types, untypedcon stant 78

UDPsocket219

uint type 52

uintptr type 52, 354, 357

unar y op erator + 53

unar y op erator - 53

unbuffered channel 226

undeﬁne d behavior 260

underly ing array 84, 88, 91, 187

underly ing typ e 39

Unico de

co de point67

es cap e 68, 107

repl acementcharac ter ( 70, 98

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

380 INDEX

st and ard 2,27, 52, 66, 67, 69, 97

unicode package71

unicode.IsDigit func tion71

unicode.IsLetter func tion71

unicode.IsLower func tion71

unicode.IsSpace func tion93

unicode.IsUpper func tion71

unicode/utf8 package69

unidirec tion alchannel typ e 230, 231

union, discriminated 211, 213, 214

univers e block46

Unix domain socket219

unmarsh alingJSON110

unnamed str uct typ e 163

unnamed var iable 34, 88

unreach ablestatement 120

unsafe package354

unsafe.AlignOf func tion355

unsafe.Offsetof func tion355

unsafe.Pointer conv ersion356

unsafe.Pointer type 356

unsafe.Pointer zerovalue 356

unsafe.Sizeof func tion354

unsig ned int eger52, 54

untypedcon stant typ es 78

unus edparameter 120

URL 123

URL escape 111

url.QueryEscape func tion111

url.URL type 193

urlvalues example160

UTF-8 66, 67, 98

UTF-8 enco dings, table of67

utf8.DecodeRuneInString func tion

utf8.RuneCountInString func tion

utf8.UTFMax value 98

value

addressable32

call by83, 120, 158

func tion132

interface 181

method 164

utf8.UTFMax 98

var de clarat ion5,30

var iable

conﬁnement 261

he ap36

http.DefaultClient 253

io.Discard 18

io.EOF 132

lifet ime35, 46, 135

lo cal 29, 141

os.Args 4

st ack 36

unnamed 34, 88

var iables, escaping 36

var iables, share d 257

var iable-size stack 124

var iadic function142, 172

ve ctor, bit 165

vendor ing 293

visibi lit y 28, 29, 41, 168, 297

visit func tion122

wait example130

WaitForServer func tion130

walkDir func tion247

we b

craw ler 119

craw ler,con cur rent239

framewor k 193

whileloop6

whit e-b oxtest311

Wi l kes, Maur ice 301

Wirth, Nik lausxiii

word example303, 305, 308

word example, test of303

worksp ace organizat ion291

wr iterlock266

wr iting effec tive tests 316, 317

xkcd JSONint erface 113

XML deco ding213

XML (ExtensibleMarku p Language)

107

(*xml.Decoder).Token method

213

xmlselect example215

zerolengt h slice 87

zerovalue

ar ray 82

boole an 30

ch annel 225, 246

func tion132

interface 182

map95

name d resu lt120, 127

numb er5,30

pointer32

reflect.Value 332

slice 74, 87

st ring5, 7, 30

st ruc t 102

unsafe.Pointer 356

The Go Programming Language

revision 3b600c, date 29 Sep 2015

From the Library of YIGUANG HU

ptg16105617

You love our titles and you love to

share them with your colleagues and friends...why

not earn some $$ doing it!

If you have a website, blog, or even a Facebook

page, you can start earning money by putting

InformIT links on your page.

Whenever a visitor clicks on these links and makes

a purchase on informit.com, you earn commissions*

on all sales!

Every sale you bring to our site will earn you a

commission. All you have to do is post the links to

the titles you want, as many as you want, and we’ll

take care of the rest.

APPLY AND GET STARTED!

It’s quick and easy to apply.

To learn more go to:

http://www.informit.com/afﬁliates/

*Valid for all books, eBooks and video sales at www.informit.com

JOIN THE

INFORMIT

AFFILIATE TEAM!

From the Library of YIGUANG HU